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Preface 



This report has been prepared, in part, to inform interested individuals, ^oups, and 
organizations regarding the Achievement Levels-Setting Project for the 1994 National 
Assessment of Educational Progress in Geography and U.S. History. The Commissioner of 
Education Statistics is authorized to "release" the data. Until the Governing Board has made 
their final decisions and these decisions have been made public by duly authorized persons, 
the achievement levels data are limited to those included in this report. Every effort has 
been made to provide informative data for review and comment. Cautions are given 
regarding the interpretation of some data, however. 

Input from various individuals, groups, and organizations is sought at tliis time because 
recommendations have not yet been submitted to NAGB for their final review and decision. 
Comment from the public will be recorded and presented to NAGB as part of the entire 
package of information to inform their decision. 

Questions and comments are encouraged and appreciated. Written comments may be 
addressed to ACT and they will be forwarded, along with other recommendations to NAGB. 

Susan Cooper Loomis, Ph.D. 
NAEP ALS Project Director 
May 27, 1995 
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Overview 



In October 1993, the National Assessment Governing Board (NAGB) awarded a contract to 
set achievement levels on the 1994 National Assessment of Educational Progress (NAEP) in 
Geography and U.S. History and the 1996 Science NAEP to American College Testing (ACT). 

The 1994 achievement levels-setting (ALS) process consisted of several steps. The planning 
and design of the process drew heavily from the experiences of the 1992 ALS process and 
evaluations of that process. Comments and suggestions solicited from various individuals, 
groups, and organizations during the design stages contributed to the final features of the 
process design. 

The planning and design stages culminated in the pilot studies for both geography and U.S 
history. The pilot studies tested all procedures under consideration for implementation in 
the achievement levels- setting process. In addition, several research studies were conducted. 
Among the most important research components was the study of rating methods and the 
differences in cutscores resulting from the different methods. Findings from those studies 
led to the choice of methods implemented in the ALS process. 

Other research issues of particular interest related to the use of more "holistic" rating 
methods and information regarding student performance relative to the achievement levels. 
These methods were called for by the National Academy of Education report.^ 

Following the pilot studies and the evaluation of the rating and evaluation data from those 
studies, the ALS panels were convened. The ALS panels for geography were convened 
November 12-16, 1995, and the U.S. historv' ALS panels were convened December 2-6, 1993. 
The results of these ALS meetings are summarized in this report. 

In addition to the design and implementation of the ALS process, however; NAGB requested 
that ACT develop additional research studies to examine the levels recommended by the ALS 
panelists. Specifically, the request was that studies be designed to address the validity of the 
ALS process and the levels set. 

ACT proposed several studies. The Technical Advisory Committer, on Standard Setting 
(TACSS) reviewed these proposed studies and recommended three for implementation. 
Priorities were assigned to the three studies, and the Similarity Classification Study was 
given top priority, followed by the Booklet Classification Study, and SlNAEP Consumer 
Survey. Very general and preliminary re^^ilts from the first two studies are presented in this 
report. The survey has not yet been conaucted. More complete information on each of these 
studies is available upon request. 



National Academy of Education Panel, Setting Performance Standards for Student 
Achievement, (Stanford Universi v. Stanford, CA.: Author. 1993) 
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Preliminary Report on the 1994 NAEP Achievement Levels-Setting Process 

for U*S* History and Geography 



Introduction 

The 1994 NAEP Achievement Levels-Setting (ALS) process was implemented November 
12-16, 1994 for Geography and December 2-6, 1994 for U.S. History. An item-by-item rating 
methodology was used by panehsts to rate the items in the 1994 item pool for each grade 
level tested by NAEP— Grades 4, 8, and 12. 

Thirty panehsts at each grade level were divided into item rating groups and items were 
divided into item rating pools. Half the panelists at each grade level rated half the items in 
the item pool for that grade. Panelists were divided into the two groups so that panehst type 
(teachers, other educators, and general public), race/ethnicity, sex, and region were 
represented as equally as possible in the two groups. Similarly, item blocks (units of 
approximately 17 items) were divided into two groups so that item characteristics such as 
difficulty (probabiUty of correct response), content type, and item format (multiple choice, 
short answer, extended response) were as equivalent as possible. 

The modified Angoff method^ was used to rate multiple choice items and the mean 
estimation method" was used to rate all other items. Panehsts were engaged in training 
exercises for approximately three days prior to the first of three item rating sessions. The 
training was designed to assure that panehsts understood the contents of the assessment 
framework and shared a common understanding of the definitions of achievement at each 
level, to help panelists reach a common agreement on "borderhne performance," to become 
familiar with the scoring rubrics and their use for each of the items in the grade-level 
assessment, and to give panelists a "reality check" with respect to student performance on 
constructed response items. 

Panelists completed a process evaluation at the end of each day or after completing a stage 
in the process. Those evaluations ;vere reviewed each day to monitor any potential problems 
or difficulties for panehsts. Respon.>es to some of the evaluation items are included in this 
report.. 

Following the final round of ratings, panehsts were provided "consequences data" that showed 
estimates of the percentages of students scoring at or above each level. A questionnaire was 
designed to collect opinions of panehsts regarding the correspondence between these 
estimates and their own expectations as to the consequences of the levels they had set. 



The procedure panehsts were instructed to follow: think of the description of what students 
should know and be able to do at the (Basic/ProficienL^Advanced > achievement level; focus on the 
minimum level of performance required to reach that level of achievement; estimate the percentage 
Df students who are at the lower borderline of the achievement lev(^l who would answer the item 
correctly. 

To rate essay items (short answer or extended response items), panelists were instructed to 
follow the same procedure outlined for the modified Angoff ratings on multiple choice items except 
they estimated the average score on each item for students at the lower borderline. 
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Panelists 

The panelist selection process is described more completely in the Design Document." 
Principles of sampling were used for drawing stratified random samples of school districts 
from a national database. The samples were stratified to represent the four NAEP regions 
approximately equally, districts wit'i enrollments of at least 50,000 students, and districts 
with at least 25% of the populatior below the poverty level. A total of six samples were 
drawn for identifying ALS nominators: one sample each for U.S. history and geography from 
which nominators of panelists of the three types (teacher, other educators, and general public 
representatives ) were identified. 

Nominators of teachers were district superintendents, leaders of teacher organizations, or 
head persons of private schools. In addition, state social studies curriculum directors were 
invited to nominate teachers from any district within their state. ^ Nominators of the other 
educators included various persons who were career educators but not themselves K-12 
classroom teachers. Nominators of the general public included the mayor (or other chief 
elected official of the primary city in the school district), the school board president, and the 
chair of the education committee of the local Chamber of Commerce. 

Organizations such as the National Council for History Education, National Coimcil for 
Geography Education, and the Geography Alliance were invited to contact nominators and 
recommend panelists. The Geography Alliance participated extensively in this process^. 

A total of 221 persons were nomina'-ed as geography panelists and 225 as history panelists 
to fill 90 positions on the panel for each NAEP subject. Panelists were selected to serve at 
a specific grade level, although they were occasionally nominated at more tl;ian one grade 
level. Panelists were nominated to represent a specific perspective: grade-level classroom 
teacher, other educators, and general public. Table la shows the distribution of the nominee 
pool for geography and Table lb shows the distribution for the U.S. history nominee pool. 

ACT project staff developed a computerized process for selecting panelists fi^om the pool of 
nominees. Panelists were rated according to their qualifications (information provided on the 
nomination form). Panelists with the highest qualifications ratings had the highest 
probability of being selected, other things being equal. The selection program was set to yield 
panels with 55% of the members representing grade-level classroom teachers, 15% 
representing other (nonteacher) educators, and 30% representing the general pubhc. The 



'* American College Testing, Design Document: Setting Achievement Levels on the 1994 
National Assessment of Educational Progress in Geography and in (7.S. History and the 1996 
National Assessinent of Educational Progress in Science (Iowa City, lA: Author, 1993). 

^ Only social studies curriculum directors from states for which one or more school districts had 
been dra\^Tl in the "teacher nominator sample" were in\ated to nominate teachers in their state as 
panelists. Unlike other nominators of teachers, these persons could nominate teachers from the st4.ite 
"at large," rather from specific districts drawn in the sample. 

The Geography Alliance state coordinators were sent letters to inform them of the persons who 
were identified as nominators in their stat^. The state coordinators were urged to contact the 
nominator and provide suggestions of "outstanding teachers" to be nominated as panelists. 

ERIC : u 
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distribution by panelist type was only one requirement to be met, however. The selection was 
set to yield panels with 20% "minority" racial/ethnic representation, up to 50% males, and 
regional representation approximately equal to the proportion of the U.S. population in each 
of the four NAEP regions. 

Some of the persons who were selected could not serve at the time required. Data for the 
actual panelists who participated in the ALS process are presented in Tables 2a and 2b for 
geography and U.S. history, respectively. Summary information ab( ut "outstanding" 
cr. dentials for panelists are included in Figure la for geography and Figure lb for U.S. 
history. 

Achievement Levels-Setting Process 

Panelists selected through the previously described process were convened in St. Louis, 
Missouri for a five-day ALS process. The same agenda was followed for both geography and 
U.S. history. During the opening sessions, panelists were provided a complete overview, via 
a computerized presentation, of the process and the procedures to be followed. Each panelist 
was given a Briefing Booklet that described each task to be performed during each session, 
the purpose for the task, and how to perform the task.^ In addition, n "glossary" of terms 
was included to help panelists with new and jargon terms. 

During the first day, a presentation by NAGB staff described the NAEP program, including 
an explanation of the government agencies involved, the contractors, and the 
interrelationships of each. That presentation traced the development of the NAEP for the 
specific subject. During the first day, panelists were administered the NAEP exam and 
allowed to self-score the exam. 

Each grade-level panel was led by a process facilitator and a content facilitator. With the 
exception of the fourth grade U.S. histor>^ content facilitator, all staff participated in both the 
pilot studies as well as the ALS meetings. Content facilitators were selected from the most 
highly recommended members of the Framework Panel for each subject. Process facilitators 
were trained extensively both before the pilot studies and before the ALS meetings. Content 
facilitators were included in at least one training session with process facilitators prior to the 
pilot study. 

Content facilitators provided training in the Frameworks and the preliminary achievement 
levels descriptions. They took the lead in all sessions to review achievement levels 
descriptions. Tniining in the Framework and preliminary descriptions was concentrated 
during the first two days of the process. PaneUsts were engaged in exercises designed to give 
them experience in working with the achievement levels descriptions. A session was 
scheduled to discuss the achievement levels descriptions, and an opportunity was provided 
for paneUsts to modify the preliminar\' descriptions before each round of ratings. 



' Tht' Briefing Booklet for geography was distributed at the start of the ALS meetings. Fur 
history, it was distributed as part of the advanced materials sent to each paneHst approximately two 
weeks pnor to the meetings. Other advanced materials sent to all panelists included a Summary 
Design Document that had been recommended as a "user friendly description of the proce durinfi: 
public comment sessions held during the spring of 1994. 
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The exercises and training provided to panelists were designed to help them become more 
familiar with the items in the NAEP for their grade level and the scoring rubxics for the 
items, as well as to become familiar with the Frameworks and descriptions. Prior to the first 
round of ratings, panelists were asked to review student responses written to all 
essay/constructed response questions. This exercise w^as especially designed to provide 
panelists with a "reality" check on student performance, to allow them to become more 
familiar with the scoring rubrics for these items, and to help them form a concept of 
borderUne performance for each achievement level. 

Instructions for each exercise and t?jsk were provided in general sessions so that all panelists 
were given the same instructions. Process facilitators repeated the instructions and 
answered questions from panelists in the grade-level sessions. All tasks were performed by 
panelists in grade-level sessions. 

Three rounds of item ratings were collected. Each panelist rated each item in his/her item 
pool (half the items for the grade). The item ratings were entered and analyzed on site. 
These item ratings were used to produce feedback iiiformation for paneUsts regarding their 
ratings relative to those of others in their grade, information about overall student 
performance on each item, information about student performance on the test booklet form 
that panelists had been administered at the start, and information about the consistency of 
their ratings on specific items. 

The feedback was presented and panelists were instructed in the meaning of the feedback 
and how to use it in subsequent rounds of ratings. Time was provided for panelists to ask 
questions and discuss the feedback before beginning another round of ratings. Feedback from 
t] ? previous round was updated after each round of ratings and presented to panelists along 
With new information. Following Round 3, all feedback was again updated relative to the 
ratings for this, the final round. 

Achievement Level Cutscores 

The cutscores in Tables 3a (geography) and 3b (U.S. history^ are reported on the ACT NAEP- 
like scale. This scale is linearly related to the NAEP scale, which means that the niunerical 
levels and relative differences obser\'ed on this scale are proportional to values on the NAEP 
scale. The approximate proportion of score points reqtiired for students at each achievement 
level is included in Tables 3a and 3b as well. These data are provided only because actual 
performance data cannot be released at this time. These data reporting the proportion of 
possiule points for each achievement level are pecuUar to the items on this exam, the sample 
of schools, and the students assessed. If the assessments were administered again, the 
percentages would change. These proportions cannot be compared from one administration 
to another. Nonetheless, they do provide some information regarding the relative levels in 
a metric that is more easily understood than the "ACT NAEP-like" scale. 

The "percent of possible points correct" data show that students needed to get at least one- 
third of the possible points for performance at the Basic level. The percentage increases 
across each grade for each subject. At the Proficient level, students needed to get nearly two- 
thirds of the possible points. Finally, at the Advanced level, students needed more than 
three-quarters of the total possible points on the items included on the assessment for the 
grade. 
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In order to get a better understanding of what these data mean, it is helpful to compare the 
percent of possible points at each cutscores to the percent of possible points scored by "the 
average" student who took the NAEP at each grade level. The percent of possible points at 
the mean for each grade on the U.S. history exam is lower than on the geography exam. This 
suggestfi that ihe U.S. history assessment was harder and /or that students performed less 
well, on average, than was the case for the Geography NAEP. B'or the Geography NAEP, 
students at the mean for Grade 4 got 42% of the tot:>J possible points on the items in the 
assessment, students at the mean for Grade 8 got 46% of the total possible points, and for 
Grade 12 students it was 48%. For the U.S. History NAEP, students at the mean for the 
Grade 4 assessment got 337c of the total possible points over all the items, students at the 
mean for Grade 8 got 38%, and students at the mean for Grade 12 got 35% of the total 
possible points on the items in the assessment for that grade level. 

A comparison of the percent of total possible points at each cutscore to the percent of total 
possible points at the mean gives a sense of the relative distribution of student performance 
at each grade level. Clearly, the proportion of students whose scores correspond to 
approximately 75% of the total possible points on the Grade 12 U.S. history assessment is 
likely to be less than the proportion of students whose scores correspond to approximately 
75% of the total possible points on the Grade 12 geography assessment, for example. 

The cutscores at each grade remained relatively stable across the three rounds of ratings, 
while the variability (reported as the standard deviation) among raters decreased. One 
generally expects the panelists' understanding of both the rating task and of the achievement 
levels descriptions to increase throughout the process. One indicator that this was occurring 
is the decrease in variability among panelists' ratings. 

Additional Analyses 

The preliminary analyses have been focused on issues of interest or concern that emerged in 
relation to the 1990 and 1992 ALS processes. 

Item Type 

Significant differences in cutscores for dichotomous (multiple choice) items and polytomous 
(constructed response/essay) items were found. In all ca^es, the cutscores for polytomous 
items are higher than for dichotomous items. With the exception of Grade 4 Ba^ic 
Geography, there is a statistically signaificant difference between polytomous and dichotomous 
item ratings. 

Panelist Type 

No significant differences were found runong the cutscores set by teachers versus nonteachers 
versus general public panelists. This finding holds for both the geography and U.S. history 
ALS panelists. 

While there is no significant difference in the levels set by panelists of the different types, 
teachers generally set the lowest levels for geography and non-teachers the highest. For U.S. 
history, the non-teachers again generally set the highest cutscores, but there is less 
consistency regarding the lowest levels. In any case, the differences are not significant. 
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Achievement Levels Descriptions 

Panelists were trained in the framework and meaning of the preUminan- achievement levels 
descriptions during the first two-three days of the ALS process. In addition to this thorough 
study of the preliminary descriptions, panelists were given opportunities to re\asit the 
descriptions and make modifications before each round of ratings. After each session of 
review and proposed modification, the content staff met to discuss the suggestions and to 
determine jointly whether the suggestions seemed in keeping with tne Framework, and 
whether the changes represented significant substantive changes. All changes for all grades 
were reviewed by all content staff If the content staff agreed that the changes wt re 
consistent with the Framework, the changes were incorporated into the achievement levels 
descriptions and then taken back to the panelists for final agreement before each round of 
ratings. Panelists were advised that once the third (final) round of item ratings had begun, 
no further changes in achievement levels descriptions could be made. 

Yery few changes were recommended. In geography, only a few words changed at each the 
three giade levels. The only substantive change was at Grade 8 Basic with the addition of 
"and their physical features." 

More changes were made in U.S. history than geography, but still the changes were few. The 
preliminary descriptions in U.S. history seemed less carefully worded than for geography. 
The content resource staff for history were clearly less committed to the wording of the 
preliminary descriptions than were the geography content staff. The changes appear to have 
improved the meaning of the descriptions and to have brought greater structural integrity 
across the grades. 

The complete text of descriptions (along with selected items to illustrate each) for each grade 
level is included for geography and for U.S. history as Figures 2 and 3, respectively. 
Panelists have recommended that the achievement levels descriptions be made more "user 
friendly" in format. They recommend that some highlighted or "bulleted" text be extracted 
from the achievement levels descriptions to make the key aspects of the descriptions clear to 
readers. 

Exemplar Items 

Exemplar items were selected from the entire item pool at each grade level. Blocks had been 
identified for release in each content area. Project staff complied with the request of NAGB 
staff to have the exemplar items selected from all item blocks— not hmited to those specific 
blocks "marked" for release by the National Center for Education Statistics and the 
Educational Testing Service, the NAEP contractor. 

Exemplar items reviewed by panelists were pre-selected using statistical criteria. The 
statistical criteria were recommended by the Technical Advisory Committee on Standard 
Setting (TACSS). The procedure was first implemented during the U.J', history pilot study 
in August, 1994. Only items having an average probability of correct response of at least 50% 
for scores within the achievement level range were included in the list of items submitted to 
ppnehsts for their consideration. Items were Hsted at the lowest achievement level for which 
the statistical criterion was met. 
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Constructed response items, scored for partial credit with 1-3 or 1-4 score points, were treated 
as separate items. For example, a score of "2" on one item could have met the statistical 
criterion for classification at the Basic level, whereas a score of "3" would have met the 
criterion from classification at the Proficient or Advanced level. Each "correct"^ score point 
(2, 3, or 4) was evaluated to determine whether the average probabihty of that score point 
was >509c within the numerical range of each achievement level. ^ 

From this list of items meeting the statistical criterion, panelists were asked to review items 
and to veto or eliminate those that would not serve to illustrate the knowledge and skills 
associated with performance for each achievement level. Because there were many items to 
review within each grade level, table groups of 5 panelists each were assigned one or two 
blocks of items to review. The table groups then made recommendations to the entire grade 
group. Any panelist wdthin the grade group could recommend additional items to eliminate 
or to "reinstate." Items could be recommended as exemplars for a higher level, but not for 
a lower level. This means that an item that met the statistical criterion for consideration at 
the Basic level, for example, could be recommended as a Proficient level exemplar item — or 
even as an Advanced level exemplar item. An item that met the statistical criteria for 
consideration at the Proficient level could not, however, be recommended as a Basic level 
exemplar item. 

Tables 4a and 4b present the counts of items on the pre-selection list meeting the statistical 
criteria and the counts of items recommended by panelists at each grade level. A few of the 
items recommended as the best of those marked for release ai'e presented in Figures 2 and 
3 along with the achievement level descriptions for which the items serve as "exemplars." 

Panelist Evaluations 

Understanding of Achievement Levels Descriptions 

Analyses of the process evaluation questionnaires provide more information regarding the 
panelists' confidence in and satisfaction with the process. Responses to questionnaires were 
reviewed on site, and no process problems were observed. Panelists generally seemed to gain 
greater confidence in and satisf^.ction throughout the process. The data reported in Figures 
4a and 4b show that their understanding of the definition of the achievement levels increased 
for ratings across rounds. 



Constructed response items that are "inappropriate" are coded "1." Those responses were not 
evaluated for consideration as exemplar items. 

A programming error led to the omission of all "items" with the highest scores from the list of 
items from which panelists made their recommendations. This error was not detected until March 
1994. Panels representing one-third of the panelists at each grade level and for each subject were 
convened May 21-22 in Iowa City. After a brief reorientation to the achievement levels descriptions 
and PVameworks, these panelists reviewed all items in the item pool for their grade, reviewed the 
entire set of items that had been recommended during the achievement levels-setting meetings in 
November and December, and then reviewed those items recommended for the "released" blocks. They 
made recommendations regarding the items that had been previously omitted. 
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Conception of Borderline Performance 

The percentages of panelists who reported that their conception of borderUne performance 
was more than moderately well formed were generally lower than the percentages who 
reported that their understanding was more than somewhat clear. Figures 5a and 5b show 
that the conception of borderUne performance gained clarity across rounds. 

General public paneUsts were less Ukely than other panelists to have gained a well formed 
conception of borderline achievement prior to the third round of ratings. The general pulpUc 
panelists in geography were less likely to report a very well formed conception of borderline 
performance for each achievement level for ratings in Rounds 1 and 2. By Round 3, their 
responses indicated that their conceptions of borderUne performance were about as well 
formed as those of other panelists. 

Similarly, the U.S. history general public panelists responded less often that their conceptions 
of borderline performance were very well formed until after the second round of ratings. By 
the second round, their reports were simUar to those of other panelists regarding their 
conception of borderline performance. 

Rating Methods 

The conceptual clarity and ease of applying increased for both the modified Angoff method for 
rating multiple choice items and the mean estimation method for rating constructed response 
items. While panelists reported increases in both dimensions for rating items of both 
formats, the conceptual clarity and ease of applying the modified Angoff method slightly 
exceeded the mean estimation mpthod throughout the process. See Figures 6a and 6b for 
responses regarding the conceptual clarity of the methods, and Figures 7a and 7b for the 
responses regarding the ease of applying the methods for item ratings. 

Differences in Performance Estimates by Item Type 

Differences in achievement levels that would be set for ratings on multiple choice items only 
versus constructed response items only were consistently observed for rating data collected 
during the 1992 ALS process for mathematics and reading. And, the pilot study results for 
geography and U S. history revealed \ae same pattern. Given those observed differences, 
questions were constructed to collect information from paneUsts regarding their opinions 
related to student performance on multiple choice and constructed response items. 

Responses indicated that paneUsts were more Ukely to attribute differences in performance 
ratings on the two types of items to diiferences in rating methodology in the beginning round 
of ratings. By the final round of ratings, however, they were more likely to disagree with this 
cause for differences in performance ratings than with differences in student behavior and 
performance on the items as the cause. This suggests that as panelists gained understanding, 
clarity, confidence, and so forth in the rating process, they became more convinced that such 
differences must be due to factors other than the rating methods. 

Across each round of ratings, panelists were highly likely to agree that constructed -response 
items assess dimensions of knowledge and skills that are significantly different from, those 
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assessed by multiple-choice items. Across the rounds of ratings, the responses tended to shift 
toward the totally agree response. 

In the final "overall" process evaluation, panelists indicated that the instructions were clear 
and their level of vuiderstanding was adequate. The average response on a 5-point scale for 
panelists in both subject ALS processes was at least 4. The amount of time given to complete 
the tasks was about right. At least three-quarters of the paneUsts in both geography and 
U.S. history also indicated that it was more than somewhat likely that they were using the 
same concept of borderline performance as other panelists for all levels of achievement. Most 
panehsts were more than somewhat confident in their ratings, and they were equally 
confident that the process was effective. About half of the panehsts (51% for geography and 
48% for U.S. history) reported that diiring discussions they feit a need to defend the ratings 
they had given to some extent. Few felt coerced to modify ratings from previous roxmds. Most 
felt that the process provided them an opportunity to use their best judgment in rating items 
to set achievement levels. Further, most felt that the process produced defensible and 
reasonable achievement levels and that the panelists were representative and credible. 
Responses to items related to the above-mentioned factors averaged around 4 points or higher 
on a 5-point scale. 

Consequences Data 

Following the fijaal round of ratings, all feedback data were updated and reported to 
panelists. In addition, data on the consequences of their ratings were provided. The 
consequences data were estimates of the percentages of students who would score at or above 
each achievement level cutscore at each grade. Panelists at each grade level were asked to 
comment on the data specific for their grade. 

No such information had been collected from panehsts diiring the 1992 ALS process. When 
the data became available to relate rating data to student performance on the NAEP scale, 
there was no way of knowing how the panelists would have reacted to the results or 
consequences of their ratings. These data were presented to panehsts for their evaluation 
to inform decisions that would ultimately need to be made regarding the achievement levels. 
The data presented to panelists were not exact, and panehsts were informed of this fact. The 
data were estimates based on a normal distribution rather than the actual distribution of 
scores on the NAEP scale. 

In general, the consequences data were consistent with panelists' expectations. Responses 
indicated that 89% of the geography panelists and 72% of the U.S. history panelists felt that 
the data reflected their expectations about the proportions of students at their grade level 
whose NAEP score would be at or above the cutscore for each achievement level. Five of the 
nine geography panelists who had not expected the results would change one or more of the 
achievement levels. In general, those panelists would opt to raise the cutscore in order to 
reduce the percentage of students scoring at or above a level. The Basic level was where 
most (4) felt the percentage should be smaller, and 2 of the 4 responses were for Grade 4 
Basic. 



• » — 

The NAEP scale distribution data were not available until April 1995. These data can only 
reported by the Commissioner of Education Statistics, or some person authorized by the Commissioner. 

erJc < ^ 
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For U.S. history, most panelists ( 13 = 579c of the 23 who did not expect those results) did not 
feel that the achievement levels should be changed in order to adjust the percentages of 
students scoring above the achievement levels. Of those 13 panelists who would make some 
changes, the general recommendation would be to lower the outpoint to increase the 
percentage of students scoring at or above the level. This recommendation was made for all 
levels at Grades 4 and 8. Only one person recommended one change in the 12th grade levels, 
and that change would be to raise the cutscore in order to reduce the percentage of students 
scoring at or above the Basic level. 

Conclusions 

The achievement levels-setting process implemented for both the Geography NAEP and the 
U.S. History NAEP went very smoothly and according to plan. The success of the meetings 
was in large part attributable to the fact that complete pilot studies had been conducted for 
each subject. 

The change that on-site staff recommended following the ALS meetings was that panelists 
not be asked to read student papers for all constructed response items in their rating pool. 
Staff felt the primary purpose of that task could be met by having panelists review far fewer 
papers leaving time to discuss the papers more fully with other panelists. Otherwise, no 
changes in the process appeared to be warranted. 



A O 
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Validation Research Studies 
Similarity Classification Study 

Overview 

The Similarity Classification Study (SCS) is a research study involving teachers and their 
students. Eighth grade teachers^^ who served as panehsts in either the pilot study or the 
achievement levels-setting (ALS) meetings for U.S. history and geography were asked to 
participate in the study. Students in up to two classes of each of these teachers were to be 
involved in a special assessment. The purpose was to determine whether and how well 
teachers who had participated in the setting of achievement levels could apply those 
descriptions of achievement to the abilities and likely performances of their own students. 
Having teachers who were very familiar with the assessment framework, the assessment 
item pool, the achievement levels descriptions, and students taking a course in the subject 
provided a best case test. These teachers would be asked to estimate the similarity of their 
students' knowledge and skills in the subject (geography or U.S. history) to the descriptions 
of what students should know and be able to do at each achievement level. 

Teachers sent the roster of students in their classes slated to participate in the study. The 
total number of students for the geography teachers was 965, and 853 students were in the 
designated classes of the U.S. history teachers. 

Students were administered a special form of the NAEP for the subject. This special, longer 
form was developed for this research study in order to get a reliable and valid measure of 
student ability over the entire content of the NAEP, as described by the Framework. The test 
form administered was twice the length of the NAEP. 100 minutes of cognitive assessment. 
Student background data and other student questionnaire data (courses in the subject, 
motivation on the exam, and so forth) were collected £tS well. 

Teachers 

Teachers were convened for their input March 4-6 in Kansas City, Missouri. Nineteen 
teachers of geography and 16 teachers of U.S. history participated in the study^^. The 
teachers were provided a brief re-orientation to the framework and achievement levels 
descriptions for their subject. The content facihtators who had taken the lead in presenting 
the frameworks during the pilot and levels-setting meetings were present to provide this 
instruction. 

Training for the task included, but was not limited to, a review of all items and the scoring 
rubrics in the eighth grade assessment. Teachers were told that the assessment to be 
administered to their students would be twice the length of a National Assessment, i.e., 100 
minutes; but they were not told the number of item blocks nor the items to be administered. 



Seven geography teachers taught at grade 7 and two at grade 9. A total of 355 students were 
in the 7th grade and 126 in the 9th. 

. The total number of geography teachers who ser\'ed as eighth grade panelists in both the pilot 
study and the ALS was 31. The total number of U.S. history teachers who served as eighth grade 
panelists in the pilot study and the ALS was 26. 



ERIC 
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Teachers were instructed that they would estimate the similarity between each student and 
the three achievement levels. They were told that there would be two such similarity 
classifications, and that specific training and instructions would be pro\ided for each session. 
During the first similarities classification session, they estimated the achievement of their 
students with respect to the framework and achievement levels descriptions of what students 
should know and be able to do. During the second similarities classification session, they 
estimated the achievement of their students vv^ith respect to the framework and achievement 
levels descriptions (as in the first classification), and they were instructed to take into 
account what they knew about the students and about the NAEP to be administered in 
making estimates about the students' performance. 

Spaces were provided for describing factors taken into account in the second classification^^. 
Most frequently mentioned factors were related to student beha\dor. The most frequently 
mentioned factor was the positive motivation of the student: this student tries hard; will take 
the test seriously and try; very focused. The high academic skills and abilities of the students 
were mentioned as often as unpredictable; immature; can do well if tries. Lack of motivation 
was the third most frequently mentioned factor: does not try hard; will not /may not ^ake this 
test seriously were comments offered to describe student behavior influencing the teacher's 
estimate. 

Students 

The students were administered an "expanded" version of the NAEP for the subject. 
Students in all 35 schools were assessed, and scoring of the data was completed, as 
scheduled, by the end of April. Of the 1803 students to be assessed, 1649 were actually 
assessed. ^'^ A total of 154 students were absent and 10 students were "excluded," according 
to the usual NAEP administration procedures. 

The expanded version of the NAEP included four blocks of items that were recommended by 
ETS, the NAEP contractor for test development, scaling, and most aspects of the assessment. 
The item blocks were selected to provide "equivalent" coverage of the NAEP item pool content 
and to be as similar as possible to the characteristics of the complete item pool, while 
minimizing the amount of time needed to test the students. 

Additional Information 

Teachers were asked to complete the teachor questionnaire and the school principal (or other 
appropriate administrative official) was asked to complete the school questiormaire. The data 
reported in these questionnaires will be helpful in comparing the relationships between 
student performances in this special study and that for the National Assessment. 



'''^ Six student forms were identified, and the teachers were instructed to describe the factors taken 
into account when estimating the achievement level to represent the performance for each of the six 
students. Two students classified at each achievement level (Basic, Proficient, and Advanced] were 
identified for these comments. 



Some of the students assessed were not included on the roster^of students submitted by the 
teacher. Thus, the final coimt of students for which complete data, i.e., both teacher estimations of 
performance and actual performance data, will be somewhat less than this number of students 
assessed. 
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Findings 

Only preliminary findings are available at this time. Comparisons of the two classifications 
by teachers revealed that teachers were more likely to estimate performance on the first 
classification as higher than on the second. That suggests that teachers believed that the 
ability of their students was higher than the students' performance on the special form of the 
NAEP was likely to reveal. Table 5 shows that only 11 percent of the second classifications 
by the geography teachers and 11 percent of the second classifications by the U.S. history 
teachers were higher than the first classifications. Geography teachers were most likely to 
make no change in the two classifications, while U.S. history teachers were most likely to 
lower the level for the second classification taking account of student behavior and the 
information available to them about the special NAEP. 

No student performance data are available at this time. The expectation is that teachers 
estimated student ability and student performance to be at a higher level of achievement 
than test performance would indicate. The computations of individual student scores and 
comparisons of those scores to the teacher's classifications has not yet been completed. 
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Booklet Classification Study 

Overview 

The Booklet Classification Study (BCS; is conceptually similar to the Similarity Classification 
Study (SCS) described above. Rather than estimating the similarities of "real" students to 
the descriptions of what students should know and be able to do at each achievement level, 
however, the estimates were made of the similarity of the performance of a student on a 
KAEP test booklet to the achievement levels descriptions. Panelists did not know the 
students who took the test; they could only read the responses the students gave to the 
questions administered on the particular booklets. 

The task was for panelists to classify the booklets into one of the achievement levels, based 
on an evaluation of the performance of that booklet relative to the descriptions of 
achievement at each of the three levels — ^plus the implicit "Below Basic" level. 

Panels 

The Booklet Classification Study was conducted for both geography and U.S. history in 
St. Louis, April 8-11. Ten panelists were invited to participate at each of the three grade 
levels for each subject. Approximately six teachers, two nonteacher panelists, and two 
general public members were represented on each grade-level panel. 

Panelists for this study were selected on the same basis as panelists for the pilot studies and 
achievement levels-setting (ALv^) panels. Previously-selected panehsts who had been unable 
to serve on either the pilot or AJLS panel were again asked to participate in this study. 
Additional persons v/ith the same qualifications as the ALS panelists were recruited to serve 
in order to fill the panels wit] i ten members each. 

The Process 

The training provided to panehsts was very similar to that proWded to the ALS panehsts. 
The first two days of orientation, training, and practice exercises were modified only slightly 
from the sessions for the ALS process. 

Staff were the same as for the ALS process, as well. Two content experts who had served 
with the previous pilot and ALS panels worked with these panelists to help them understand 
the frameworks and to gain a confident imderstanding of the achievement levels descriptions. 
Process faciUtators who served in both pilot studies and ALS meetings again led panelists 
through the training and the process. 

The Task 

The task was for panelists to classify 40 NAEP booklets into the three achievement levels — or 
the "Below Basic" level. The booklets had been carefully selected according to statistical 
criteria. Namely, booklets were selected for which at least 4 of the 5 plausible values were 
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within the achievement level range. A special effort was made to reduce the number of 
different items with which panelists would need to become familiar in classifying booklets. 

The criteria used for selecting blocks of items for this study were the same used for the 
similarity classification study. Namely, the item characteristics for the items in the blocks 
selected matched as closely as possible those for the complete NAEP item pool. The 
distribution of items with respect to content categories and item formats were also 
approximately equal. Given that, ^he minimum number of different booklet forms, i.e., 
combinations of two blocks of items containing the four blocks identified for the study, were 
selected for the study. Specific booklets meeting the performance criteria (4 of 5 plausible 
values at the level) were then identified from among the booklets of each form. 

Panelists generally had no difficulty completing the task wdthin the allocated time. All 
history panehsts had completed the task within approximately five hours. The geography 
paneUsts did not work so rapidly, and the last panelist completed the tasks after over seven 
hours. 

Findings 

The analyses will be conducted to determine the correspondence between booklet 
classifications and student performances. There is no expectation that there will be a perfect 
correspondence between the two. As indicated in the proposal for the study, however, 
"evidence in support of the achievement levels would be a reasonable level of relationship 
between the classification based on the NAEP scale data and the classifications by panelists, 
and a low level of shift in the assignment to categories." A "low level of shift" means, for 
example, that the panelist-s classified the booklets within one achievement level of the NAEP 
scale classification. 

Only vei-y preliminary analyses of the data have been conducted at this time. Based on these 
analyses, it appeairs that panelists generally classified the booklets at a level lower than the 
empirical (plausibb" values) classification. This finding would indicate that the achievement 
levels were set too iow. 

Further studies of the actual booklets used in the BCS are underway. One hypothesis is that 
panelists were unable to treat items in the way they are treated in NAEP scaling. This 
would mean that treatment of items in the model used by panelists for classifying booklets 
was different than that in the Item Response Theory model used for scaling NAEP data. 
Data collected from paneUsts on the strategy' they used for the task, observation on site, and 
results indicate that many — if not most — panelists used a non-compensatory model and that 
they .did not treat "not reached" items as if they were not administered. These considerations 



The instructions from our Technical Advisors were to select booklets with 3 of the 5 plausible 
values within the achievement level range and the average of plausible values at least as great as 
that for the level. If those criteria could not be reached for the targeted number of booklets per level, 
then fewer booklets were to be presented to the panelists. Tht targeted distribution of booklets was 
7 at Below Basic, 13 at Basic, 13 at Proficient, and 7 at Advanced. The target of 7 booklets . t the 
Advanced level, given the various criteria, was not met for all grades. The "rule" was that no te^-er 
than 5 booklets could be used for the Below Basic or Advanced levels. Additional forms were to be 
added to assure that a minimum of 5 booklets were included for each level. Thus, for some grades, 
fewer than 40 booklets were presented to panelists for classification. 

^/ 1 
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would lead panelists to classify performances represented in tooklets as lower than the IRT 
scaling would be likely to do. Until further research can be completed, however, this is only 
speculation. 
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Table la 
Geography ALS Nominee Pool 





TR 


NT 


GP 


Total 


Grade 4 


59 


8 


12 


79 


Grade 8 


50 


6 


12 


68 


Grade 12 


49 


9 


16 


74 


Total 


158 


23 


40 


221 




(727f ) 


(10%) 


(187c) 





Grade 4 
Grade 8 
Grade 12 



Male 



White 



Is 



26 



In 



24 



Female 



}vIinoritv 



Ratings 
2's 



18 



27 



19 



Total 



Total 



3's 



35 



26 



31 



Grade 4 


13 


66 


79 


Grade 8 


34 


34 


68 


Grade 12 


37 


37 


74 


Total 


84 


137 


221 






(38'''c) 


(62%) 







Grade 4 


74 


5 


79 


Grade 8 


1 ^ 


10 


68 


Grade 12 


1 6^ 


10 


74 


Total 


196 


25 


221 




( 89^r 1 


(119c) 
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Table lb 
U.S. History ALS Nominee Pool 





TR 


NT 


GP 


Total 


Grade 4 


29 


12 


20 


61 


Grade S 


37 


17 


20 


74 


Grade 12 


43 


15 


26 


84 


Total 


109 


44 


66 


1 

219 j 




(50%) 


{ 20% ) 


(30%) 





Male Female Total 



Grade 4 


15 


46 


61 


-1 

il 

J 


Grade 8 


35 


39 


i 

74 


Grade 12 


46 


38 


84 1 


Total 


96 
(44%) 


123 
( 56% ) 


219 j 

i 




White 


Minority 




Total 




Grade 4 


42 


19 


61 ! 


Grade 8 . 


60 


14 


1 


Grade 12 


74 


10 


84 


Total 


176 
(80%) 


33 
(20%c) 


219 




I's 


Ratings 
2s 




3"s 




Grade 4 


20 


20 


19 




(.;rade 8 


26 


26 


22 


Grade 12 


29 


36 




19 


Total 


75 
(34%) 


82 
(37':; ) 


60 
( 27'.f 1 
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Table 2a 



Grade 4 

Grade 8 
Grade 12 

Totai 



TR 



Geography Panelists 

NT 



17 



18 



18 



53 
(607c) 



12 
(14%) 



GP 



Total 



24 
(277c) 



30 



28 



31 



89 



Is 



Ratings 
2's 



3's 





Male 


Female 


Grade 4 


6 


24 


Grade 8 


11 


17 


Grade 12 


13 


18 


Total 


30 


59 




(34%) 


(66%) 




White 


Minority 


Grade 4 


27 


3 


Grade 8 


24 


4 


Grade 12 


27 


4 


Total 


78 


11 




{887c) 


(12%) 


Northeast 


16 


18% 


Southeast 


25 


28% 


Central 


22 


25% 


West 


26 


29% 



Grade 4 


2 


5 


23 


Grade 8 


2 


6 


20 


Grade 12 


2 


8 


21 


Total 


6 


• 19 


64 




(7%) 


(21%:) 


( 72% ) 
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Table 2b 
Ij\S, History Panelists 





TR 


NT 


GP 


Total 


Grade -1 


14 


5 


/ 


26 


Grade 8 


14 


4 


t 


25 


Cirade 12 


16 


4 


6 


26 


TotaJ 


44 


13 


20 


77 




(57%) 


(17%) 


(26%) 


■ 



I's 



Ratings 
2"s 





Male 


Female 


Grade 4 


n 

y 


1 7 


Grade S 


6 


19 


Grade 12 


13 


13 


TotaJ 


28 


49 




(36%) 


(64%) 




White 


Minority 


Grade 4 


19 


7 


Grade 8 


20 


5 


Grade 12 


23 


7 


TotaJ 


62 


15 




(81%) 


(19%) 


Northeast 


16 


21% 


Southeast 


21 


27% 


Central 


21 


27% 


West 


19 


25% 



3's 



( Jrade 4 


5 


9 


12 


Grade 8 


2 


13 


10 


Grade 12 


1 


13 


12 


Total 


8 


35 


34 




(10%) 


(45%) 


(44%) 
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Table 3a 



Geography ALS Cutscores"^ and % Correct Data'' 





Basic 


Proficient 


Advanced 


Grade 4 


Round 1 


149.0 


166.8 


178.9 




(9.9)'-'' 


(4.2) 


(3.9) 


Round 2 


148.2 


166.5 


178.6 




(7.9) 


(4.3) 


(4.0) 


Round 3 


148.3 


166.5 


178.6 




(7.5) 


(4.3) 


(4.0) 


Ic CoiTect 


33.3% 


61.1% 


80.8% 


Grade 8 


Round 1 


144.7 


162.5 


174.4 




(5.5) 


(5.3) 


(7.1) 


Round 2 


148.1 


163.9 


176.4 




(5.5) 


(4.6) 


(5.7) 


Round 3 


148.6 


164.4 


177.3 




(5.9) 


(4.7) 


(5.4) 


Correct 


37.0% 


60.7% 


79.9% 


Grade 12 


Round 1 


145.8 


164.4 


176.8 




(9.9) 


(7.3) 


(8.6) 


Round 2 


148.2 


165.7 


182.0 




(5.7) 


(6.1) 


(5.8) 


Round 3 


147.7 


165.5 


181.9 




(5.2) 


(5.8) 


(5.3) 


9f Correct 


38.0% 


63.6% 


83.6% 



Cutscores are on the* ACT NAEP-like sjale. 

^ correct data are estimates of the percentage of p(' sible pnints required for a score at the 
lower borderline of epch achievement level. Read: "A student would have to get at least 
of the possible points on the items to score at the Advanced level in Grade 12." 

Numbers in parentheses ar^.» standard deviations for the cutscores. 
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Table 3b 

U.S. History ALS Cutscores'^ and 9^ Correct Data^ 





Basic 


Proficient 


Advanced 


Grade 4 


Round 1 


157.1 


170.1 


iso^e 




(7.3) 


(3.8) 


(5.1 ) 


Round 2 


155.1 


168.4 


177.8 




(4.6) 


(3.4.) 


(4.9) 


Round 3 


155.5 


168.8 


178.3 




(4.8) 


(3.4) 


(4.6) 


9c Correct 


33.1% 


56.7% 


75.2% 


Grades 


Round 1 


155.2 


171.5 


185.1 




(6.8) 


(4.1) 


(6.8) 


Round 2 


154.2 


171.1 


184.5 




(5.1) 


(3.9) 


(5.7) 


Round 3 


154.3 , 


171.2 


184.6 




(5.5) 


(3.7) 


(6.2) 


9f Correct 


37.19c 


61.69t 


80.7% 


Grade 12 . ■ 


Round 1 


159.3 


170.7 


182.5 




(9 0) 


(3.1) 


(5.2) 


Round 2 


159.6 


171.3 


182.9 




(6.2) 


(2.5) 


(4.4) 


Round 3 


159.6 


171.6 


183.6 




(4.9) 


(2.6) 


(4.0) 


9(- Correct 


41.57c 


63. 27^ 


82.39c 



Cutscores arc on the ACT NAEP-Hke scale. 

'•^^ 9r correct data are estimates of the percentage of possible points required for a score at the 
lower borderline of each achievement level. Read: "A student would have to get at least 83.69r 
of the possible points on the items to score at the Advanced level in Grade 12." 
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Table 4a 



Geography Exemplar Items 
from Released Blocks Only 





Statistically 


Panelists 




vSelected 


Recommended 


Oradp 4 






Basic 


26 


21 


Proficient 


16 


14 


Advanced 




Q 


Grade 8 






Basic 


31 


4 


ij 

Proficient 


22 


6 


Advanced 


9 


2 


Grade 12 






Basic 


44 


9 


Proficient 


13 


4 


Advanced 


6 


2 


Grade 12 (w/Q3G3) 






Basic 


56 


19 


Proficient 


19 


7 


Advanced 


9 
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Table 4b 



U.S. History Exemplar Items 
from Released Blocks Only 





Statistically 
Selected 


Panelists 
rveconirTic nueu 


Grade 4 






Basic 


26 


5 


Proficient 


25 


20 


Advanced 


S 


7 


Grade 8 






Basic 


32 


18 


Proficient 


IS 


10 


Advanced 


11 


/ 


Grade 12 






Basic 


46 


18 


Proficient 


2] 


16 


Advanced 


6 


5 
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Table 5 

Changes in Classifications Within and Across Levels: SCS 



Subject 


Change"^ 


Frequency 


Percent 


Geography 


Negative 


301 


31 




No Change 


555 


58 




Positive 


101 


11 


U.S. History 


Negative 


419 


50 




No Change 


324 


39 




I Positive 


94 


11 



33 



■ ' Changp -• I Classification 2 - Classification 1) 
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Figure la 

Outstanding Credentials for Geography Panelists 

Teachers 

National Awards 

1994 Teacher of the Year (State Farm Insurance Company) 
National Park Service Outstanding Educator 
President's Award for Science Teaching 
Geography Education Award 
WTio' Who in Teaching 
Unspecified (3 J 

State Awards 
Teacher of the Year (5) 

Governors Awards/Citations for Teaching Excellence (2) 
Montclair Award (CBS.) 
Other/unspecified (7) 

Geography Alliance 

23 Teachers 

6 Nonteacher Educators 

Nonteacher Educators 

Local School/District Administrators (7) 
State Administrators ( 1) 
Post-Secondar\' Faculty (5) 

General Public 

Mayor 

Owner map store 
State Engineer 
Owner, IVucking Business 
Director of State Division of Archaeology- 
Peace Corp; Korean experience 

Geography major and President of a broadcasting company 

VISTA volunteer; Human Resources; well-traveled 

District Manager, worldwide communications company 

City Planner and Director of Community Development Programs 

Retired Military; much travel 

Research Analyst; well-traveled 

Retired Program Manager for Lockiieed; well-traveled; ties to education 
Architectural Historian; travel; cultural geography interests and trainin, 
Dairy farmer with 5 children; geography is special interest 
Military; world traveller 
CEO small credit unit 
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Figure lb 

Outstanding Credentials for U.S. History Panelists 

Teachers 

National Awards 

DAR History Teacher of the Ycai' 

FuUbright to Turkey 

XEH Fellowships (3) 

U.S, Inst, of Peace Grant 

Certified grants writer (secured a $5 million grant) 
WTio's Who Among Teachers 

Consulting work for Georgia State Department of Education 

Woodrow Wilson Fellowship for Anierican Histoiy at Princeton University 

Unspecified 

State Awards: 

Teacher of the Year (4) 

Citations for Teaching Excellence (2) 

Ai SS Teacher 

Chairperson of Curriculum Assessment m AL 
Governor's Award for Science 
Other/unspecified (2) 

Memberships: 
NCSS (14) 
State level CSS (8) 
Other (5) 

Nonteacher Educators 

Local/District Administrators (6) 
State Administrators (4) 
Post-Secondar}^ Faculty (4) 

General Public 

Author (2) 

Park Ranger; New England & SW History 
Pediatric Occupational Therapy 
Owner, video production; western history 
Museum Director 
Oregon Trail history' 

Computer Network Coordinator; Constitutional Historv* 
Business Manager, Capital Electric Corp.; World Wars & Space Programs 
Owner, mortgage company; plans on masters in secondary education 
Museum Director; degree in history, UCal-San Diego 

Bookroom clerk at junior high; helps students and school with book selections 

Supervisor, Air Traffic Tower; Historical Society 

Executive Director, Chamber of Commerce; education 

Child Welfare Administrator; education; school board; tutoring 

Museum Director; local historj-; historical reading 

Executive Director, Chamber of Commerce; history interest 
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Church Secretar>'; school board; antiques; writes about antiques 

Executive Director, Historical Society; Ph.D. Amer. Studies, Hist. & Envm. History 

Museum Curator; US History; French/Indian War; \VW II; Scotish & Colonial histon.' 

Executive Director, Greater East St. Louis Chamber; world history 

Owner, TMS Services; school board; education; history 

Executive Director, Historical Society; college archeolog>'/anthropology 

Asst. Federal PubUc Defender; Political science major; student teaching in history 

Memberships: 
NCSS{8) 
Other ( 1 ) 
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Figure 2 

Geography Achievement Levels Descriptions 
and Exemplar Items 



I 4th-Grade Basic — Students should be able to use words or diagrams to define basic 
, geographic vocabuiar>'; identify personal behaviors and perspectives related to the 
: environment and describe some emdronmentai and cultural issues in their community; use 
i visual and technological tools to access information; identify major geographic features on 
! maps and globes; be able to read and draw simple maps, map. keys and legends; 
\ demonstrate how people depend upon, use, and adapt to the enviromnent; and give 
; examples of the movement of people, goods, services, and ideas from one place to another. 
' In addition to domonstratmg an understanding of how individuals ai;e alike and different, I 
they should demonstrate a knowledge of the ways people depend on each other, j 




ERLC 



Grade Level: 4 

Achievement Level: Basic 

Block Number: Q1G3 

Item Number: 11 >3 

Overall P-Value: 77 
Avg Conditional P- Value: 59 

Score Levels: 3 

Content Area: 1 
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Rationale Text: 
Scoring Guide 

Scoring Rationale: Student locates and correctly labels the North Pole, South Pole, and 
the equator on a map showing the Western hemisphere. 

3 - Complete. The response correctly labels all three features on the map. 

2 - Partial. The response correctly identifies the location of one or two features on 
the map. 

1 - Inappropriate. The response does not correctly locate any of the three features. 



Credited Responses: 

- the North Pole at the top of the map 

" the South Pole at the bottom of the map 

- the BcTuator in the mc^'s center. 



■I 
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Questions 12-13 are based on the highway map below. 



BO001961 




MAJOR HIGHWAYS 



Highway route, 
matnty expressway 



12. The map shows that one part of the country has more major 
highways than the other part of the country. Why is this? 

• There are more people and cities in the eastern part of the 
country. 

(D It IS easier to build highways in the eastern part of the country. 

© Cars are not an important form of transportation in the 
western part of the country. 

(o) States are larger in the western pan of the countr>^ 



BO001962 



Grade Level: 4 

Achievement Level: Basic 

Block Number: Q1G6 

Item NuniDer: 12 

Overall P-Value: 60 
Avg Conditional P-Value: 59 

Score Levels: 2 

Content Area: 3 
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4th-Grade Proficient— vStudents should be able to use ft\ndam.ental geographic knowledge ; 
and vocabulary' to identify basic geographic patterns and processes; describe an j 
environmental or cultural issue from more than one perspective: and read and interpret I 
information from visual and technological tools such as photographs, maps and globes.' 
aerial photography, and satellite images. They should be able to use number and letter | 
grids to plot specific locations; understand relative location terms; and sketch simple maps | 
and describe and/'or draw landscapes they have obsei-\'ed or studied. Proficient students ! 
should be able to illustrate ways people depend upon, adapt to, and modify the 
em-ironmeni: describe and/or illustrate geographic aspects of a region xising fundamental 
geographic vocabulary and give reasons for current human migration; discuss the impact 
a location has upon cultural sunilarities and differences; and be able to demonstrate how 
a i event in one location can have an impact upon another location. 



Questions 6-7 refer to the followmg chan. 



MAJOR EXPORTS OF THREE COUNTRIES . 



Countn' A 


1 

Countr\' B 


Countrs' C 


' Oil 

i Natural Gas 
1 Coconuts 

i 


Cars j 
Televisions , 
Cameras j 


Computers ; 
Airplanes 

Wheat ; 

1 



KI000759 



6. The situation shown in the chart %viU probablv lead to 
m trade among all three countries 
® trade only between countries A and B 
© trade only between countries B and C 
© a decision by each country- to produce all nine goods listed 



v;.iooo760 



Grade Level: 


4 


Achievement Level: 


Proficient 


Block Number: 


Q1G6 


Item Number: 


6 


Overall P-Value: 


37 


Avg Conditional P-Value: 


67 


Score Levels: 


2 


Content Area: 


3 


••••• V |- ' 
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countries sun'ounding the Persian Gulf .ire important to many other 
countries in the world? 
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Grade Level: 4 

Achievement Level: Proficient 

Block Number: Q12G7 

Item Number: 6 

Overall P-Value: 34 
Avg Condrtional P-Value: 63 

Score Levels: 2 

Content Area: 3 
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MIDDLE EAST 6 




Working oil field 
C ^ Known oil deposit 



Almost 73 of the world's 
petroleum is produced by OPEC* 
OPEC members include the 
Persian Gulf countries of 
Iran. Iraq, Kuwait. Qatar, 
Saudi Arabia, and the United 
Arab Emirates. Other OPEC 
members are Algena. Libya, 
Nigeria. Gabon, Indonesia, 
Ecuador, and Venezuela. 

CHANGING BOUNDARIES 




LEADING PETROLEUM PRODUCERS 



mm 



OPEC^ 



•Organization of Peiroieum Exporting Countries 




Mediterranean 
Sea 




80 km 
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Israel occupied the area shown in dark green 
-I1 1967. Alter the Six Day War of that 
"tr. it also controlled the parts of Egypt. 



Mediterranean 
Sea 




80 km 



In 1975. 1979. and 1982. Israel withdrew in 
stages from the Egyptian territory known as 



the Smai Peninsula. 



dan and Syria shown in light green. J^J^^rj^ COPY AVAILABLE 
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Sinai 
\ Peninsula 



/ SALTDI 
ARABIA 




80 km 



After withdrawing from the Sinai. Israel 
remained in control of the Gaza Stnp. West 
Bank, and Golan Heights. In 1981 Israel 
claimed the Golan Heights as pan of its state 
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Item Number: 6 Keyr.NOKK Accession Number: BO001942 
Classification Codes: 

1/2 III B APPLY SA WRIT ATLAS 

Look at the map on the top of page 65* What is a reason yrhy the countries surrounding 
the Persian Gulf are important to many other countries in the world? 



Rationale Text: 
Scoring Guide 

Scoring Rationale: Student demonstrates ability to interpret map of middle Kast. 

3 - Complete. The response mentions the fact that the countries surrounding the Persian 
Gulf region are important because of the number of oilfields located there. 

1 - Inappropriate • The response does not identify any reasons why the Persian Gulf is 
important to the rest of the world . 



40 
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: 4thX;rade Advanced— vStudents should be able to use basic geographac knowledge and 
vocabuhirv' to descnbe global patterns and processes; describe ways mdividuals c:in protect 
and ouliance environmental quality; describe how modifications to the environment may 
have a variety of consequences; explain differing perspectives that apply to local 
environmental or cultural issues: and demonstrate an understanding of forces that result 
in migration, changing demographics, and boundary changes. They should be able to solve 
simple problems by applying information learned through working with \isual and 
technological toois such as^ aerial and other photographs, maps and globes, atlases, news 
media, and computers. They should be able to construct models and sketch and label maps 
of their owm state, the United States, ^md the world; use them to describe and compare 
differences, similarities, and patterns of change in landscapes; and be able to predict the 
impact a change in one. location can have on another. • They should be able to analyze the 
ways individuals and groups interact. 



WAYS TO GET RID OF WASTE 

. Dumpmg tar out m the ocean 

. Burning 

• Recycling 

. Burymg in landfills 

From the list above, select one method of gettmg rid of waste and 

identify one advantage and one disadvantage of this method. kiooc626 

Method of waste disposal: ■ — — 

Advantage; ~ 



Disadvantage: 



ERIC 



Grade Level: 4 

Achievement Level Advanced 

Block Number. Q1G6 

Item Number: 11 ^3 

Overall P-Value: 33 

Avg Conditional P-Value. 80 . . 

Score Levels. 3 

Content Area: 2 
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Item Number: 11 Key 



NONB 



Accession Number: KJ000S26 



Classification Codes: 
1 II B 



APPLY 



SA 



WRIT 



WAYS TO GET RID OF WASTE 



# 



# 



Dunping far out in the ocean 

Burning 

Recycling 

Burying in landfills 



From the list above, select one method of getting rid of waste and identify one 
advantage and one disadvantage of this method. 



Method of waste disposal: 
Advantage : 



Disadvamtage : 



Rationale Text: 
Scoring Guide 

Scoring Rationale: Student demonstrates ability to identify one advantage and one 
disadvantage of a selected waste disposal method. 

3 - Complete. The response accurately describes an advantage and disadvantage of one 
method of waste disposal. Explanations should be both specific to that method and 
geographically logical. Correct answers may be drawn from the list given or include 
socne other appropriate response. 

2 - Partial. The response describes either an advcuitage or a disadvantage. Correct 
answers may be drawn from the list given or include some other appropriate response. 
If present, the other description is incorrect or trivial, as in, "dximping waste in 
oceans has no effect on us,* or ^•it takes a long time to dvmp waste in oceans". 

1 - Inappropriate . The response does not accurately describe an advcintage or a 
disadvantage of a given method of waste disposaJL. 
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Credited Responses could include: 

Ocean Dxiiopina 

Advantages 

- Easy, low cost solution 

- No costly land wasted for landfills 

r No unsightly incinerators and landfills 

- Requires no technology 

- It can't be seen 

Disadvantages 

- Kills sea life (fish, birds, plants) 

- Destroys fishing industry (especially shellfish) 

■ Ruins tourism (beaches closed for health and safety reasons) 

- Water contamination 

■ Not all things are recyclcible 

Btiminq Waste 
Advajitaaes 

- Reduces cost and waste of land used for landfills 

- Energy created as a byproduct (steam and electricity) 

- Recycling of useful products often accompanies burning 

- can be economical option (in some cases, low waste transportation costs) 
Pi sad vctnt ages 

- The production of ash and toxic gases 

- Air pollution 

- Costly to regulate and check satfety of process 

- Difficulty finding a location for incinerator 

- Opposition to siting 
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Recycling 
Advantages 

- Produces less pollution than other methods 

- Preserves natxiral resoxirces (examples include wood, minerals, fuel) 

- Reduces waste levels 

- Reduce cost and waste of Icind used for landfills 

- New uses for discarded items 

' increases public awareness of waste and pollution 

- encourages manufactures to produce 

D i sadvan t age s 

- Achieving the motivation and cooperation req^oired 

- Requires public education and enforcement 

- Need to find a market for recycled products 

- Not all things axe recyclaUDle 

- Needs initial equipment that may be expensive (economic factor) 

- Other types of pollution may be produced during recycling 

■ Burying Waste 
Advantages 

- Cheap, although only in the short term 

- Simple 

- Low technology 

- In areas with low population density, can be popular with p\ibli.c 
D i sadvant age s 

- High costs of transporting waste to distant landfills 

- Shortage of affordable and available land 

- Contamination of water, soil, and air 
" Smell 

- Opposition to siting 

- Not all products are biodegradable 
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7. Look at the population and landscape map of China on page 72 of the atlas. 
What area of the country is densely populated (crowded)? 



Give two reasons why this area of the country' is densely populated. Bocowi^ 



Grade Level: 4 

Achievement Level: Advanced 

Block Number: Q12G7 

Item Number: 7 >3 

Overall P~Value: 20 

Avg Conditional P-Value: 79 

Score Levels: 4 

Content Area: 3 

48 
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ThP nrpat Wall of China was built over 2.200 years ago as a barrier 
^^\^Mor>gol invaders. It remains the world's longest structure. 
P R I (J \g across 1.500 miles (2415 km) of northern China 



49 



Acjnculture 

pi?c^r rvM>v /\niF 



CHINA'S 
LABOR FORCE 
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Item Number: 



7 Key: NONE 



Accession Number: BO001944 



Classification Codes: 
1/2 III A 



UNDER 



EA 



WRIT 



ATLAS 



XjQOk at the population and landscape map of China on page 72 of the atlas. 
What axea of the country is densely populated (crowded) ? 



Give two reasons why this area of the country is densely populated. 



Rationale Text: 
Scoring Guide 

Scoring Rationale: Student displays an ability to interpret a population and landscape 
map of China by stating which area of the country is densely populated. The student also 
demonstrates an understanding of the causes of population density. 

4 - Con5)lete. The response correctly identifies an area of population concentration 
{the east coastal area or Sichxian Basin) . It eaqjlains this by drawing on two of the 
explanations given in the list below or gives some other appropriate response. 

3 - Essential* The response correctly identifies cin area of pop\ilation concentration 
(east coastal cirea or Sichuan Basin) . It explains this by stating one of the 
explanations given in the list below or gives some other appropriate response. 

2 - Partial. The response correctly identifies an area of population concentration, but 
does not correctly explain why the population is concentrated in certain areas in 
terms of the explanations given or some other appropriate response, 

1 - Inappropriate. The response does not correctly identify an cirea of population 
concentration as the east coastal area, or the Sichuan Basin, and does not explain 
the population patteim in terms of the explanations given below or some other 
appropriate response. 



Credited Responses could include: 
Influences on Population Distribution 

- Coastal areas have flat land, suitable for building and agriculture 

- The coastal area allows trade with other countries 

- Oceans and rivers cire in^ortant for settlement with a reason such as transportation, 
agriculture, household use, industry, fishing 

- Much of China is mountainous or desert and is not suitable for settlement 

- Government administrative center; capital (also major cities) 

- the area has a lot of business and industry 

- the area has a lot of agriculture 
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8th-»C;rade Basic — ^Students should possess fundamental i-aiowledge and vocabulary- of 
I'oncepUi relating to patterns, relationships, distance, direction^ scale, boundan.-, site, and 
ivituation; solve fundamental locatioiial questions using latitude and longntude; interpret 

! simple map scales; identify continents and their physical features, oceans, imd various 
countries and cities: respond accurately to descriptive questions usms: information obtained 

1 by use of visual and technological tools such as geographic models and/or translate that 

1 information mto words: explain differences between maps and globes: and find a wide range 
of information using an atlas or almanac. Students should he able to recognize and 

, illustrate the relationships that exist between humans and their ennronments, and provide i 
e%ndence showing how physical habitat can influence human acti\ity. They should be able ! 
to dofmc a region and identify its distinguishing characteristics. Finally, they should be i 
able to demonstrate how the interaction that talces place between and among regions is ! 
related to the movement of people, roods. 5er\nces. and ideas. 



iSTIMATTLD WORLD OIL RESER\TES, 1986 




Saudi 
Arabia 



Kuwait 



Soviet 
Union 



Iran 



Mexico 



United 
States 



.1000691 



1. According to the graph above, the largest oil reserv'es in 1986 were in the 
® Gulf of Mexico 



® Canbbean region 
• Persian Gulf region 
® Gulf of Guinea 



K1000691 
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Grade Level: 8 

Achievement Level: Basic 

Block Number; Q2G4 

Item Number: 1 

Overall P-Value: 85 
Avg Condrtionai P-Value: 88 

Score Levels: 2 

Content Area: 1 
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Questions 3^ are based on the map below showing the distribution of earthquake 
epicenters around the world between 1961 and 1967. 




3. Between 196 1 and 1967, the area that had the most earthquakes was the 
<2> Mediterranean basin 
<D mid-Atlantic Ocean 
© Caribbean Sea 
• Pacific Ocean rim 



Grade Level: 


8 


Achievement Level: 


Basic 


Block Number: 


Q23G6 


Item Number: 


3 


Overall P-Value: 


84 


Avg CorxJrtional P-Value: 


89 


Score Levels. 


2 


Content Area; 


1 



52 
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8th-C;rade Proficient— ^^tudents should possess a fundamental geographic vocabulan-, 
undc-^rstand geogi'aphy s analytical concepts; solve locational questions requiring in teijration 
of information from two or more sources, such as atlases or globes; compare information 
presented at different scales: identify a wide variety of physical and cultural features and 
: describe regional patterns. Students should be able to respond accurately to interpretive 
' questions using geography's visual and technological tools and translate that information 
I into patterns; Identify differences in map projections and select proper projections for 
! vanous purposes; and develop a case study working with geography s analytical concepts, 
i In addition, students should be able to describe the physical and cultural characteristics of 
piace5^; explain how places change due to human activity; explain and illustrate how the 
concept of regions can be used as a strategy for organizing and understanding Earth's 
surface. Students should be able to analyze and interpret data bases and case studies as 
j well 05 use information from maps to descnbe the role that regions play in mfluencing 
i trade and migration patterns and cultural and political interaction. 



8. The maior areas or wheat production in the world are the central United 
States and Canada. Ukraine, south central Australia, and the pampas or 
Argentma. What is the characteristic shared bv these areas that explains 
their role m wheat production^ 

O All have rainy, damp climates. 



® All are near sea coasts. 



• AI 



1 are plains. 



® All are in highland regions. 



Grade Level: 
Achievement Level: 



Block Number: 
Item Number: 
Overall P- Value 



8 

Proficient 
Q23G6 



Avg Conditional P-Vaiue 



Score Levels: 
Content Area: 



8 

52 
76 
2 
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Quest^i ons 16-17 refer to the map below 



SE0C06?^ 




16. This map would be most useful to a. 

® pilot flying from Europe to South America 
• pilot flying fn :n Canada to Scandinavia 
® person sailing to Antarctica 
® person sailing in tropical seas 



DO001953 
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Grade Level: 8 

Achievement Level: Proficient 

Block Number: Q2G4 

Item Number: 16 

Overall P-Value: 46 
Avg Conditional P- Value: 80 

Score Levels: 2 

Content Area: 1 
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l'r<.limHi:ir\ l^-port tm th<- TJlM AL< Pn)ct-.> tor T S. Hi-^.torv ( H-oirr^plu 



8th-(;rade Advanced— St uclonts should h:m- a coram:md exten^:fivo s.^eop'aphic 
knowUxl^f-. analytical concepts, and vtKabulary; tie able to anaivzp spaiial phenomona using 
a variety of sourcrs with iaformation prcsontcd at a variety of scaios and show 
re!ationshipj> between them: and use case studies for spatial analysis and to develop maps 
and uihcr ^ji^aphics. Students should be able to identify patterns of climate, vegetation, and 
population across Earth s surface and interY)ret relationships between and amonj: these 
pattenis. ^ind use one category' of a map or aenal photosri^aph to predict other features of 
:i place such as vej?etation based on climate or population density iiased on topographic 
features. Studcnts'^should also be able to relate the concept of region to specific places and 
explain how rej^ions change over time due to a variety of factors. They shfjuld be able to 
profile aVeeion. of their own design using geographic concepts, tools, and skills. 



3. V.'hat would a scientis: probablv scudv ic predict where acid ram wviuld 
tali- 

•X The atomic biructurcs o: suiiur. niiroccn arri oxvi:cr. 

C?> rWass-transit svstems that serve maior ci::es 

• Wind patterns that prevail over maior manuiacturini: areas 

© The location ot sewace-treacment plants Kioooti 



Grade Level: 8 

Achievement Level: Advanced 

Block Number: Q2G4 

Item Number: 3 

Overall P-Value: 36 
Avg Conditional P-Value: 84 

Score Levels: 2 

Content Area: 2 
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4, What are the conditions that make the tundra difficult for human settlement? 



Grade Level: 8 

Achievement Level: Advanced 

Block Number- Q2G4 

Item Number: 4 

Overall P-Value: 75 
Avg Conditional P-Value: 62 

Score Levels: 3 c 

Content Area: 1 '^^^ 
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Item Number: 4 Key: NONE Accession Number: KJ000824 
Classification Codes: 

2 II B UNDER SA NA NA 

What are the conditions that make the tundra difficult for humaii settlement? 



Rationale Text: 
Scoring Guide 

Scoring Rationale: Student demonstrates knowledge of characteristics of tundra. 

3 - Complete. The response identifies at least two characteristics of the tundra and 
may explain how these -aake it difficult for human settlement. 

2 - Partial. "The response identifies only one characteristic of the tundra and may 
ejqjlam how this makes life difficult for human settlement. 

1 - Inappropriate. The response does not identify any characteristics of the tundra o- 
e^laxn how they make it difficult for human settlement. Or it provides answers that 
are inappropriate. . *- ^ 

Credited Responses could include: 
Tundra 

- climate inhospitable 

- very wet 

- very cold 

- permafrost 

- frost almost all year 

• lack of vegetation to support survival 

- long hours of darkness 

- lots of mosquitoes in summer 

- lack of precipitation (generally low precipitation) 

- icy gale- like winds 

NB: Do not accept: - land is dry 

- weather is bad 

- snow 




S7 
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"l2th-(;rade Bii-sic— Students should rxissess a knowledge of concepts and u^rms commonlv 
used in phvsicid and human Reoffraphy as well :,ls skills enabling them t--. oniploy applic^able 
units of measurement and scale when solving simple localional problums usmg maps and 
-blies. They should be able to read maps; pro\nde examples of phuns. plateaus, hills, and ■ 
mooniains; Imd locate continents, major bodies of water, ;md selected countries and cities. 
They should be able to inten.iret -eographic data and use visual and technoiugictil to-!s 
such ai charts, tables, cartogi-ams, :md graphs; Icnow the luiture of and be able to lacntify 
several basic types of map projections; understand the basic physical structure of the : 
i planet: explain and apply concepts such as continental dnft and plate tectonics; and! 
' describe i^eo^'raphy's anaUticai concepts using case studies. Students should have a ' 
i comprehensive understanding of spatial relationships including the ability to recognize ; 
I patterns that exist across Earth in terms of phenomena, including climate regions, time j 
i zones, population distributions, availabihty of resources, vegetation zones, and | 
1 transportation and communication netv/orks. They should be able lo develop data bases 
I about specific places and provide a sunple analysis about theu- importance. 




3. On the map above, the shaded countries represent the n:embership 
of the 



Organization of Petroleum Exponmg Countries lOPEC. 
® World Health Organization (WHO- 
© Nonh Atlantic Treaty Organization (NATO) 
CD British Commonwealth of Nations 



Grade Level: 


12 


Achievement Level: 


Basic 


Block Number: 


Q3G4 


Item Number 


3 


Overall P- Value: 


65 


Avg Conditional P-Value: 


62 


Score Levels: 


2 
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Questions 3^ are based on the map below showing the distribution of earthquake 
epicenters around the world between 1961 and 1967, 




: All earthquake epicenters, 1961 - 1967 



4. What is responsible for the pattern of eanhquake activity shown on the 
map? 

® Volcanic eruptions 

® The weight of ocean water pressing on the land 

© Hurricanes and cyt .>nes 

• The movement oi tectonic plates 



Grade Level; 


12 


Achievement Level: 


Basic 


Block Number: 


Q23G6 


Item Number: 


4 


Overall P-Value: 


75 


Avg Conditional P- Value: 


79 


Score Levels: 


2 


Content Area: 


1 



59 



BEST COPY AVAILABLE 



Preliminary Report on the 1994 ALS Process for U.S. History and Geography 



Page 52 



Rationale Text : 
Scoring Guide 

Scoring Rationale: Student demonstrates ability to locate four features on a map showing 
Europe^ Asia, Africa, and Australia. 

4 - Complete. The response correctly labels all 4 features on the map. 
3 - Essential, The response correctly labels 3 features on t:he map. 
2 - Partial, The response correctly labels 1-2 feature on the map. 

1 - Inappropriate. The response incorrectly labels all of the features on the map. 

N3: If students write out answers instead of putting the numbers on the map, give credit 
for correct response. 



Item Number: NONE Key: NONE Accession Number: KJ000479 
Classification Codes: 

3 NA NA NA NA NA TEXT 



A dense canopy of interlaced broad- leafed trees blocks the t 
and shades the forest floor all year in this sparsely populated 
area. Shallow- rooted trees that rise 150 feet or more have 
broad trunks with support roots above groxind. Rain occurs 
every day and the forest floor is always daiip and dark. 



Rationale Text: NONE 



I'r- iHinnarN K.'port «>n tlw- 



U Al 



Pi* 



for r S Hl.■^tu^^ :iiui ( ifiLTaptw 



Pace 



12th-(Jrade Proficient— Students should havr- -.m extensive- underst;in.dini: and knowledge 
t)t ihi: concepts :md trrmmotoicv r.f physical aiid human geography. They sh(^uld be able to 
ust: ^icopraphic concepts \n ari:i}yze spatial phenomena and to discuss cccaiomic. political, 
and sochil factors that dcfmo and interpret space. They should be able. to do this through 
tho interpretation oi maps ;md other visual and technological toi.ls, through the analysis 
ot case sttidies. the utihzation of data bases, and the selection of appropriate research 
materials. Students should be able to design their own maps based on descriptive data: 
descni)e the physical and cultural attributes of major world regions: relate the spatial 
distribution of population to economic and envii-onimontal factors; report both historical and 
contemporary events within a 'j:eographic framework using tools such as special purpose 
maps, aiid primaiy and secondary source materiaJs. 



Por Question 5 wriic vou: answer on the map below. 




5. Write the number or each of the tollowing phvsical features 
m the correct location on the map on page 6 

1 Pyrenees Mountams 

2 The Japanese Archipelago 

3 Mediterranean Sea 
Persian Gulf 



ERLC 



Grade Level: 


12 


Achievement Level: 


Proficie:^.t 


Block Number: 


Q3G4 


Item Number: 


5 >3 


Overall P-Value: 


23 


Avg Conditional P-Value: 


52 


Score Levels: 


4 



Preliminary Report on the 1994 ALS Process for U.S. History and Geography 



Page 54 



Section 3 ' 



CO 



C 
o 

N 



s 

7 
6 

¥ 5 



3 
0 



5 1 

0 



TRAVEL TO EUROPE, 1948-1991 



kMlMl 



lUiUlUltl 




Year 



9. What is the dominant trend shown in the graph? 



Give two major reasons for the trend shown. 



ERIC 



Grade Level: 12 

Achievement Level: Proficient 

Block Number: Q23G6 

Item Number: 9 >3 

Overall P-Value. 94 
Avg Conditional P-Value: 55 

Score Levels: 3 

Content Area: 3 



K100052C 
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Rationale Text: 
Scoring Guide 

Scoring Rationale: Student demonstrates ability to interpret what the dominaat trend in 
a gr^h is and to give two reasons for the trend • 

3 - Complete. The response identifies the major trend in the graph and provides two. 
apprqpriate reasons to e^qplain this. 

2 - Partial. The response identifies the major trend in the graph but gives either no 
reason, or only one reason to explain t:his. Or, the response provides two appropriate 
reasons, but fails to identify the trend. 

1 - Inappropriate. The response does not indicate the major trend shown in the graph," 
but may give one reason'lEor the trend as suggested in t:he list given, it may just copy 
words from chart . 

Credited Responses may include: 

Trend in Graph 

An increase in the number of United States citizens traveling to Europe between 1948 
and 1991. 

Appropriate Reasons 

Improvements in travel - air travel is more affordable 

- air travel is faster 

- air travel is safer 
Increase in tourism - more leisure time 



Internationalization 
of economy 



more affluence 
aging population 
no major BuropccLn way 
cultural identity 
more %iranted to go 
more business travel 
more political travel 



G5 



I'ri'liiiuii;ir\ lic-uort u\\ the HV.M ALS Vrna^^^ for T S. Hisiurv and (^rotrraphv 



Papr- 5t) 



12th-(lrade Advanced— Students should possess a comprehensive understanding of 
Keojrraphic knowied^^e and concepts: apply this knowledj^e lo case studies; formulate 
hypotheses aiid ti^si geographic models that dumunstratc complex relationships between 
■ physical and human phenomena; apply a \yide range of map skills; develop maps usmg 
iundamontal carto^aphic principles includmg translatin.2: narratives about places and 
events into ^T"aphic representations, and use other visual -and technological tools to perform 
locational ^malysis and interpret spatial relationships. vStudents should also be able to 
' undertake sophisticated analysis from aerial photographs or satellite imagerj^ and other 
i visuals. Advanced students should be able to develop criteria assessing issues relating to 
! human spatial organization and en\TronmentaI stability and. through research skills and 
: the application of critical thinking strategies, identify alternative solutions. They should 
ibe able to compile data bases from disparate pieces of information imd from these data 
; develop ^ener<'ilizations and speculations about outcomes when data change. 



Mexico Cuv is an example of runaway urban growth. Every^ day an 
estimated average of 1 700 people move there trom villages m the 
countr/side. In addition, more than 1,000 babies are bom m the city 
daily Some geographers think that as manv as dO million people will 
hve'there bv the vear 2000. Thousands ot tamilies survive on the 
equivalent ot a few dollars a day, and most members of these tamilies 
have no prospects for steady jobs or much improvement in the 
physical quahtv of their lives. However, regardless ot the hardship 
and the povertv, people continue to pour into Mexico City. 



KI000401 



4. Give two reasons why people continue to move to Mexico City 
despite the difficult Uving conditions. 



SE0007S7 



Grade Level 12 

Achievement Level: Advanced 

Block Number Q3G4 

Item Number' 4 >3 

Overall P Value. 93 
Avg Condrtional P Value: 68 

Score Levels 3 

Content Area: 3 



C4 
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Item Number: 4 Key: NONE Accession Number: SE000757 
Classification Codes: 

3 III C UNDER SA WRIT TEXT 

Give two reasons why people continue to move to Mexico City despite the difficult 
living conditions. 



Rationale Text: 
Scoring Quids 

Scoring Rationale: Student -Remonstrates ability to determine why people continue to move 
to Mexico City despite its difficult living conditions. 

3 - complete. The response gives two reasons why people continue to move to Mexico City. 
The idea must be conveyed that more opportunities not guarantees are available. 

2 - Partial. The response gives one reason that explains why people continue to move 
to Mexico City. 

I - Inappropriate. The response does not' give any reason explaining why people continue 
to move to Mexico City. 



Credited Responses could include: 



Reasons for Migrating to Mexico City 

Opportunity to improve standard of living, 
facilities, or en^loyment opportunities 

Difficulty of rural conditions. Bscape 
unenployment, or low wages , or poverty 



Better housing, or education, or medical 
poor housing, or medical facilities, or 



Decline of traditional agricultiure 
Displacement of peasants 

Relatives or friends in city - provide emotional and practical support. 



ERLC 



65 
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Section 



10, Fossil fuels such as oil and coal aie formed from 

® geological processes that transform organic materials 
CD the . apid decay of animal bones 

CS> organic processes that lead to the fossilization of animal tissue 

® anificial processes used to treat and reuse garbage k]ocos52 



11. Afte^ we anchored our ships in the ocean and went ashore to explore, we 
marched west. The forest was so thick we could only travel three miles in 
the first two days. Then we came to the mountains and climbed to the top. 
A rushing river flowed west out of the mountams. We continued to march 
two miles west and came down out of the mountains. Two miles funher we 
came to the coast. It was obvious that the area we were exploring was an 
isthmus. 

In the box below, draw a map of the region described above. Be sure to include 
all of the geographical elements mentioned in the description. Include a scale to 
indicate distances. 

KICOCS^? 



1 



Grade Level: 12 

Achievement Level: Advanced 

Block Number: Q23G6 

Item Number: 1 1 ^4 

Overall P-Value: 86 
Avg Conditional P-Value: 74 

Score Levels: 4 

Content Area: 1 



ERIC 
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Rationale Text: 
Scoring Guide 

Scoring Rationale: Student demonstrates an understanding of direction, isthmus, and 
simple map making. 

4 - Complete* The response includes an accurate map in which at least 4 elements are 
correctly placed. The response must be an isthmus and have direction of travel and 
river correctly. 

3 - Essential* The response includes a map in which 3 elements are correctly placed. 
The response may be a peninsula or an island. 

2 - Partial* The response'^-.includes a map in which at least 2 elements are correctly 
placed. 

1 - Inappropriate. The response does not include a map or t±ie map shows none of the 
elements correctly. 

NOTE: No answer t:hat gets direction of map wrong can get more than a 2. Use of the 
scale is not necessary to get a 4. 

Features on Map 

Bast coast 

Forest 

Motintains 

River flowing west 

West coast 
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Fiffure 15 



U.S. History Achievement Levels Descriptions 
and Exemplar Items 



4th'Grade Basic \ 

Fourth-^rade students performm^^ at the basic level sriould he able to identify and describe ! 
a few of the most familiar people, places, events, ideas, and documents m American histor>'. | 
They should be able to explain the reasons for celebrating most national holidays, have 
some famiUarity with the geography of their own state and the United States, and be able 
to express in writing a few ideas about a familiar theme in American histor>\ 



14. Wnich area became part or the United States last- 



Hav.aii 



® Texas 



© Oregon 



CD Alaska 



BOOO 



Grade Level: 
Achievement Level: 



Block Number: 
Item Number: 
Overall P-Vaiue 



4 

Basic 
Q1H4 



Avg Conditional P-Value 



Score Levels: 
Content Area: 



14 

55 

59 

2 

4 



GS 
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Questions 11-12 refer to the statenaent below. 

A house divided against itself cannot stand. I believe this government 
cannot endure permanently half slave and half free. I do not expect the 
Union to be dissolved— I do not expect the house to fall — but I do expect 
it will cease to be divided. 



What war broke out soon after the statement was made? 
© American Revolution 
® War of 1812 

© Mexican-American War 
• Civil War 

BOOC ; 1 1 c 



Grade Level: 4 

Achievement Level: Basic 

Block Number: Q12H7 

Item Number: 12 

Overall P- Value: 54 
Avg Conditional P-Value: 58 

Score Levels: 2 

Content Area: 1 
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4th'(irude Proficient 

Fourth-;rrade students perfornung at the proficient level should be able to identify, describe 
:md commtint on the si^^nificance of many liistorical people, places, ideas, events, and i 
docume!:Us. They should interpret information from a variety of sources, including texts. ; 
maps, pictures, and timelines. They should be able to constmci a simple timeline from 
data These students should recognize the role of invention and lechnolo^cai change in . 
history. They .should also recognize the wa>'s in which sreographic and environmental * 
factors have influenced life and v^'ork* : 



7, Your teacher has asked vou to teach vour classmates about ON'E o: these 
ramous people trcm .Ajncrican histor\': 

Georcc VVashmcton 
Soioumer Truth 
Sittinq Bull 
Thomas ietfersor. 

Choose one of these tamous people. 

My famous person from American histor\- is 



Write down three facts about the person vou have chosen that would 
help you teach about that person. 



Fact I 



Fact 2 



Fact 3 



BOOOM33 



ERLC 



Grade Level: 4 

Achievement Level: Proficient 

Block Number: Q1H5 

Item Number: 7 >3 

Overall F-Value: 35 
Avg Condftional P-Value: 65 

Score Levels: 4 

Content Area: 1 



70 
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Q1H5^07 

Item Number 7 Accession Number: BO001133 



Your teacher has asked you to teach you classmates about ONE of these famous plople from 
American history': 

George Washington 
Sojourner Tnjth 
Sitting Bull 
Thomas Jefferson 

Choose one of these famous people. 

My famous person from American history is , 

Write down three facts about the person you have chosen that would help you teach about that 
person. 

Fact 1 



Fact 2 



Facta 



Rationale Text: 

1 - Inappropriate, The response gives ones of the four names but is unable to provide any 

accurate facts about his or her identity, although some attempt is made to provide facts. 

2 - Partial. The response gives one of the four names and provides one accurate fact about 

his/her identity. 

3 - Essential. The response gives one of the four names and two accurate facts about his/her 

identity. 

4 - Complete. The response gives one of the four names and three accurate facts about 

his/her identity. 

NOTE: If no name is chosen, but accurate facts are given for one of the people listed in the item, 
the response may receive credit. 

Also, facts derived from reading the question (for example, "my name is George") are 
unacceptable. 
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Credited Responses could include: 



George Washington 
RevolLrtionary War 
president 
had wooden teeth 
lived in Mt. Vernon 
survived Valley Forge winter 
help his men to survive Valley Forge 

helped to think of the plan that defeated the British at Yorktown 
led Constitutional Convention 
General in the French and Indian War (Sever-Years War) 
fought in a war 



crossed the Delaware 
owned slaves 
surveyor 
Mt. Rushmore 
on $1 bill 
on quarter 



General 

Sojourner Truth 

Famous African/American abolitionist 
freed slave 

spoke against slavery aiid for women's rights 

helped to free slaves 

speaker 



do NOT accept the cherry tree story 



Underground Railroad 
gender and wori< rights 
feminist 

travelled through New England and Western 
States spreading her views 



Sitting Bull 
Native American 

led his people in a war against the US gov't 
in the 1800*s (1874) in order to prevent 
seizing of their lands & forced removal to resen^ations 

Little Big Hom - Custer 



Sioux/chief/lndian/medicine man 
spent hears touring with a Wild West show 
escaped with his people to Canada 



Thomas Jefferson 

writer of the Declaration of Independence 
Secretary of State for Washington 
president 

sent Lewis and Clari< on their expedition 
had slaves 

Ambassador to France 



from Virginia 
on the nickel 
on the $2 bill 

inventor - made n^chines (NOT light) 



General Comments: NONE 
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. What IS the purpose of the Bill of Rights? 
® To say how much Americans should pay in taxes 
• To protect freedoms like freedom of speech 
© To describe the jobs of the President and Congress 
® To make Washington, D.C., the capital of the United States 



Grade Level: 4 

Achievement Level: Proficient 

Block Number: Q1H5 

Item Nunnber: 8 

Overall P- Value: 45 

Avg Conditional P- Value: 65 

Score Levels: 2 

Content Area: 1 

73 
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4. What IS the main reason the Pilgrims and Puritans came to America? 
• To practice their reUgion freely 
CD To make more money and Hve a better life 
© To build a democratic government 
©. To expand the lands controlled by the king of England 



BOOCiOCS 



Grade Level: 4 

Achievement Level: Proficient 

Block Number: Q12H7 

Item Number: 4 

Overall P-Value: 41 
Avg Conditional P~Value: 80 

Score Levels: 2 

Content Area: 1 



74 
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> 4th rude Advanced 



Fourth-grade students performingr at the advanced level ;;hould have a beginnmg 

; understanding of the relationships between people, places, ideas, events find documents. 

' They should know where to look for information, including reference books, maps, local 
museums, inter\^ews v.^th family and neighbors, and other sources. They should be able 
to use histoncal themes to organize and interpret historical topics, and to incorporate 
insights from beyond the classroom into their understandmg of history. These students 

■ shoiiid undersc;^d and explain the role of invention and technological change in historj-. 

^ They should also understand and explain the ways in which geographic and environmental 
factorr- have influenced life and work. 



Mavilower Arrives — 




Good Harvest 



1 1620 


i ! 


1621 


Feb. Tv'iarcJ A:Dril 


i 


! 






! Sept. 


Oci. iNov i Dec. 


Jan. 


1 


May ; )u.ie 


luly j Aug. 


Sept. 


Oct. 



•Mayflower 
Leaves 
England 



-Harsh Winter 




^Wampanoag Help 
With Crops , 




First Thanksgiving' 




ERIC 



3. What can vou tell from the time line? 

® The Mayflower took more than one year to sail to Plymouth. 

® The Mayflower arrived in Plymouth in 1621. 

• The tirst Thanksgiving was not celebrated in the same month 
as It is today. 

<© The Pilgrims in Plymouth ate mostlv fish. 



BO00097ft 



Grade Level; 
Achievemen'i Level: 
Block Number: 
Item Number: 
Overall P-Value: 



Advanced 

Q1H5 

3 

32 



Avg Conditional P-Value: 74 
Score Levels: 2 
Content Area: 1 



75 
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2. Your teacher has asked you to teach your classmates about ONE of these 
famous places where an important event m American histor>' happened: 



the Alamo 
Pearl Harbor 
Gettysburg 
Roanoke Island 

My famous place in American history* is' 



Write down three facts about the place that you have chosen that will help 
you teach your classmates about that place. 



3G00 I ! 36 



Fact 1 



Fact 2 



Fact 3 



Grade Level' 4 

Achievement Lever Advanced 

Block Number Q12H7 

Item Number: 2 >4 

Overall P-Value 81 
Avg Conditional PA/alue 62 

Score Levels: 4 

Content Area 4 
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Item Number: 



2 Accession Number: BO001136 



Your teacher has asked you to teach your classmates about ONE of these famous places 
where an important event in American history happened: 

the Alamo 
Pearl Harbor 
Gettysburg 
Roanoke Island 

Hy famous place in American history is 



Write down three facts about the place that you have chosen that will help you teach your 
classmates about that place. 



Fact 1 



Fact 2 



Fact 3 



Rationale Text: 

To determine whether students understand three important facts about one of the places 
listed. ^ 

1 - Appropriate. The answer gives no facts that are relevant to the particular place and 
that might help another person understand the place, but makes an attempt at giving 
f a c t s . 



2 - 



3 - 



4 - 



Partial. The answer gives one fact that is relevant to the particular place and that 
mighi. help another person understand the place. 

Essential. The answer f,ives two facts that are relevant to the particular place and 
that would help another person understand the place. 

Complete. The answer gives three facts that are relevant to the particular place and 
that would help another person understand the place. 



ERIC 
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Q12H7^02 

Or^dited responses could include : 

^Note - grade notations indicate information most likely to have been mastered in 4th or 8th 
grade ) 

the Alamo 

( 4th ) - Spanish mission used as fort (m San Antonio, TX ) 

-Used as fort during battle fought by band of Texans demanding independence from 
Mexico. 

-Texans defeated by Mexicans (March 1835) 
(8th) - Texans fought ( assisted by some Mexicans ) under Colonel Travis 
- Many Texans at Alamo were recent settlers from U.S. 

- held out in fort, but defeated w/ barely survivors 

- defeated by Mexicans led by General Santa Anna 

- there was a war there 

- Davey Crockett^ Jim Bowie died defending the Alamo, Sam Austin was not there 

Pearl Harbor (8th Grade) 

- in Hawaii in the pacific 

- bombed by the Japanese in 1941 , ( on December 7th ) 

- led to U.S. joining Allied forces in WWII - declared war against Japan 

- most important navel base in Pacific 

- can accept many people died 

Gettysburg - (8th Grade) 

- site of important battie in Civil War 

- in Pennsylvania, close to Washington DC 

- site where President Lincoln delivered important speech (in 1863) honoring soldiers who 

died in battle there 

- President dedicated a part of the battlefield there to a cemetery for soldiers 

- union forces defeated Confederate forces here, so badly that the battle was a tuming point 

in the Civil War 

- stopped Southem advance in Civil War 

Roanoke Island ( 8th Grade) 

- Off coast of present- day North Carolina 

- Algonquin Indians lived there before white settlers arrived 

- First English settlement there in 1585 

- Soldiers led by Walter Raleigh est. a fort there in 1585 

- soldiers there traded with Indians (1585), but retumed home because of trouble with Indians 

and food shortages. 

- second group of English settlers led by John White) brought families, and received land, in 

order to make a permanent settlement in 1587 

- this second group of settlers disappeared 

General Comments: NONE 



< O 



Prrlimm.:r\ K.-i^..r or: the IW\ A].> I'r tnr IVS. Hislor\ andUMUzrapn\ 



Hth-(irade Basic j 

Eiizhth-;:rade students perfomims at the basic level should be able to identify rmd place in 1 
coniexL a rdn^e of historical people, places, events, ideas, and documents. They should be • 
able to distinguish between prhncu y :uid secondary sources. They should have a beginning 

' und(^rptaiidiiii; of the diversity oi the jVnierican people and the ways in which people from 
a wide variety of national ^.uid cultural heritages have become part of a single nation, j 
Ei^hth-^rrade students at the basic level should also have a be^mn^^ undersU-mding of the i 
fundamental political ideas and institutions of American life and their historical origins. . 

. They should be able to explain the significance of some major historical events. | 



Que- lion-- reter to the p.ibs:izc bclou 

We ho!:: ti:esc trjths to pc s-jh-evident: That ail men are created equal, 
that :hev are endowed bv their Creator with certain unahenable riE;h:5, that 
amor.:: these are li!e iibertv .md the pursuit ot happiness. That, to secure 
•hes'.' riehts governments are instituted among m.en. deriving their lust poucr^ 
rrom :ne consent or the j;overncd. that, whenever anv torm of government 
becomes destructive of these ends, it is the right or the people to alter or to 
abolish ii and to institute a new government. 

— 1T76 



5, The priman* author oi the document was 
O George Washmgton 
C£) John Marshall 
CD Robert H. Lee 
^Thomas [efferson 



Grade Level: 8 

Achievement Level; Basic 

Block Number: Q2H4 

Item Number: 5 

Overall P- Value: 71 
Avg Conditional P-Value: 79 

Score Levels: 2 

Content Area: 1 
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Questions 1 M2 reter to the stacenient beiow. 

A house divided against itself cannot stand. I believe this government 
cannot endure permanently half slave and half free I do not expect the 
Union to be dissolved — I do not expect the house to fall — but I do expect 
It will cease to be divided. 



1 1 . The statement was made by 
® George Washington 
® Thomas Jefferson 
• Abraham Lincoln 
® Theodore Roosevelt 



BOOOI lOS 



Boccncr 



Grade Level: 


8 
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11 
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2 
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1 
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L During the 1500's and 1600 s, what was the major cause of death among 
Indians of the Americas? 

O Warfare among tribes 

® Warfare between Native Americans and Europeans 
• Infections and diseases brought by Europeans 

® Changmg climatic conditions b^-^" 



Grade Level: 8 

Achievement Level: Basic 

BlocK Number: Q23H9 

Item Number: 1 

Overall P-Vaiue: 69 
Avg Conditional P-Value: 75 

Score Levels: 2 

Content Area: 2 C: / 
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, Sth'drade Proficient I 

Eighth-srade students performing at the proficient level should be able to explain the 
siffaificance of people- places, events, ideas, and documents, and to recognize the connection 
between people and events wnthin historical contexts. They should understand and be able 
to explain the opportunities, perspectives and chailenc:es associated ^-iih a diverse cultural 
population. Thev should incorporate geographic, technological, and other considerations in 
their understanding of events and should have knowledge of significanL political ideas and 
institutions. They should be able to communicate ideas about historical themes while citing 
evidenct: from primary' and secondar>' sources to support their conclusions. 



10. What was the mam reason that manv leaders :n Great Britain ieaned 
toward supportine; the Conteaeracv m the C:v:'. War* 

G> Plantation owners in Britain heid slaves 

CD Most British immigrants to the United States lived m the South 
• Britain relied on Southern cotton tor its factories. 

® British politicians wanted to make the United States weaker so 

that they could conquer it. bocom:; 



Grade Level: 8 

Achievement Level; Proficient 

Block Number: Q23H9 

Item Number: 10 

Overall P-Value: 41 
Avg Conditional P- Value: 63 

Score Levels: 2 

Content Area: 4 



ERLC 



Preliminary Report on the 1994 ALS Process for U.S. History and Geopraphy 



Page 75 




The /X->/rut.(i.m .>< Tt.) jr b*>-ti,n HurS^r N Cumcf 
IP46. Museum nt the Citv «>? Ncu York The Hjrrs T 
Petcr> Collection 



3. Identify the event that is ponrayed in the picture above. 



Why IS the event important in United States history! 



ERIC 



Grade Level: 8 

Achievement Level: Proficient 

Block Number: Q2H4 

Item Number: 3 >3 

Overall P-Value: 97 
Avg Conditional P- Value: 55 
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Content Area: 1 
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Item Number: 3 Accession Number BO001902 

Q2H4^03 



- insert picture of Boston Tea Party 
Identify the event that is portrayed in the picture above. 



Why is the event important in United States history? 



Rationale Text: 

Students can identify the Boston Tea party from an engraving of the event and 
explain something about the event's significance. 

1 = Inappropriate. The response docs not correctly identify the event, or 
offer any information relevant to the event. 

2 = Partial. Tlie response correctly identifies the event (as the Boston Tea 

Party and does NOT just repeat the title of the picture) or explains what the 
colonists are doing without addressing its historical significance. OR, it 
describes the historical significance but does not explicitly identify the 
event as the Boston Tea Party. 

3 = Appropriate. The response correctly identifies the event as the Boston 
Tea Party. It also explains that thf*. event is important because it showed 
colonial determination to combat undesirable British policies, and to seize 
control of the colonial economy and of colonial politics. 



General Comments: NONE 



llA Zls Vl 

Identifies AND Explains Identifies OR Explains Neither identifies nor 

explains 

• ID: Boston Tea Party repeals title of 
engraving 

description 

• Explanation 

step on road to revolution 
political activity result of frustration 
regional importance - t bed of resistance 

erJc 84 
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Sth-Grade Advanced 

Eighth-grade students performmg at the advanced level should recognize significant themes 
and movements in historv and begin to understand particular events in light of these 
themes and movements. They should have an awareness of continuity and change over 
time and be able to draw relevant analogies between past events and present-day ^ 
situations. They should be able to frame questions about historical topics and use multiple 
sources to develop historical generaUzatious and interpretations. They should be able to 
explain the importance of histoncai themes, including some awareness of their political, 
social, and economic dimensions. 



6. What was one consequence of Nat Turner s rebellion- 
CD Large numbers of slaves fled to the Nonh. 
® Slave revolts broke out throughout the South. 
© Conditions for slaves on mam southern plantations improved. 

Southern states passed laws designed to tightly control slaves boocisj3 



ERIC 
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14. What goal was naost inaportant m shaping United States foreign policy 
between 1945 and 1990 ? 

• Preventing the spread of communism to new areas and weakening 
It where it already existed 

CD Encouraging trained scientists and other skilled workers who lived 
m foreign countries to irmnigrate to the United States 

© Strengthening the United States industrial and agricultural sectors 
to help them compete against the British and the French 

"S) Providing foreign aid to all poor countries to help them develop 

economically and technologically booou9c 



Grade Level: 
Achievement Level: 



8 

Advanced 
Q23H9 



Block Number: 
Item Number: 



14 

26 

85 

2 

4 
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Questions 16-17 refer to the cartoon belov;. 



HCC0)-i6: 




There's a long, long trail 
winding 



Bruce Shanks in T.nc Biiiraio News. 



17. What IS the main message of this cartoon? 



Booc tit- 



Grade Level: 
Achievement Level: 
Block Number: 
Item Number: 
Overall P- Value: 



8 

Advanced 
Q23H9 
17 >2 
11 



Avg Conditional P-Value: 69 
Score Levels: 3 
Content Area: 1 
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Itc^m Numher: 17 Accession Number: 30001464 Q^-^H; 



WJ)al.is the nunn nie.S5:age of this cartoon? 



1 - Inappropriofc. 'Jlic response docs not correctly identify the nio<5<;age of the cartoon. It does luu 

indicate thai the cartoon refers to problems in llie civj] rights moveiiicnl in the United Slater:. 

2 - Partial. The re.<;pi)nse understands tliat the cartoon is about problems associated with the civil rights 

bill but is not able to explain the carioon in a specific and preci.se fashion. 

."^ Appnipriate. 'Hie response correctly idenlines the main me^\sagc of the carlooti as being that passing 
civil right<; law.s does not mean U^at civil lights issues are resolved, and that social, economic, and 
political (and not just legal; changes wciv also necessary. 

CJenerai Comments: NQNR 
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I2th-Gradc Basw 

I 

Twelfth'gi-ade students performing at the basic le\-ei should be able to identL^^ the 
significaace of many people, places, events, dates, ideas, and documents in U.S. history. 
They should also recognize the importance of unitv' and diversity in the social and cultural 
history of the United States, and an awareness of Americas changing relationships with 
the rest of the world. They should have a sense of continuity and change in histor\^ and be 
able to relate relevant experience from the past to their understanding of contemporary 
issues. They should recognize that history is subject to interpretation and should 
understand the role of evidence in making an historical argument. 



LIVE BIRTHS PER 1,000 RESIDENTS IN THE UNITED STATES 




1920 1925 1930 1935 1940 1945 1950 1955 1960 1965 19:^0 

Year 

12. The graph above suppons which statement about the birthrate in the United 
States: 

CD It Jechned steadily from 1920 to 1950. 
® It increased rapidly during the Great Depression. 
• Ii mcreased after the Second World War. 
® It tendtid to increase after each war. 



BOCX)l573 



Grade Level: 12 

Achievement Level: Basic 

Block Number: Q3H5 

Item Number: 12 

Overall P~Value: 49 
Avg Conditional P-Value. 60 
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Content Area: 2 



89 



Preliminar>' Report on the 1994 ALS Process for U.S. History and Geography 



Page 82 



7. The Monroe Doctrine was intended to 
O promote United States trade with China 
® help keep the peace in Europe 
• discourage European involvement in the Americas 
® protect United States business in Japan and Korea 



ERLC 



Grade Level: 12 

Achievement Level: Basic 

Block Number: Q23H9 

Item Number: 7 

Overall P-Vatue: 41 
Avg Conditional P-Value: 50 

Score Levels: 2 
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12th-Grade Proficient 

Twelfth-grade students perfornuixg at the proficient level should understand particular 
people, places, events, ideas, and documents in historical context, with some awareness of 
the politicaL economic, geographic, social religious, technological, and ideological factors 
that shape historical settings. They should be able to communicate reasoned 
interpretations of past events, using historical evidence effectively to support their 
positions. Their v.xitten arguments should reflect some in-depth grasp of issues and refer 
to both primary and secondary' sources. 



"Our reconstruction measures were radically defective because they failed 
to give the ex-slaves any land." 

— Frederick Douglass 

4. Descnbe bneily the way m which Douglass's statement helps explain the nse 
of sharecropping in the South after the Civil War. In your answer, be sure to 
define the term sharecropping. booci"'3c 
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Grade Level: 12 

Achievement Level: Proficient 

Block Number: Q3H5 

hern Number: 4 >3 

Overall P-Value: 19 
Avg Conditional P-Value: 57 

Sc»re Levels: 4 

Content Area: 1 



91 



Preliminary Report on the 1994 ALS Process for U.S. History and Geography 



Page 84 



Item Number 4 Accession Number: BO001730 Q3H5_04 



-Our reconstruction measures were radtcaily defective because they failed to give the ex-slaves 

- Frederick Douglass 

li^^"^u^'if"yf^^^'''^y '^'^ Douglass's statement helps explain the rise of sharecropping in 
the South after the Civil War. In your answer, be sure to define the term sharecropping. 



Rationale Text: 

To determine whether students understand what sharecropping was and can link It to 
reconstruction measures. 

1 - Inappropriate. The response does not understand sharcropping or Reconstruction. 

lf'f^I}'f response is able to define sharecropping or is able to show an understanding 
of what sharecropping is, but does not explain Its use or its rise. 

I'o^f response accurately defines sharecropping. The response links the use of 

Sharecropping to general economic conditions faced by ex-siaves (causal), but does not link the 
use of onarecropping to the failures of reconstruction policy in general or to the failure of " 
reconstruction to bring about land reform. For example, "slaves had it rough/poor." 

ll^IJLKltK "^^ f^P°"se accurately defines sharecropping. The response explains the 
link between the nse of sharecropping and specific reconstruction policies or explains the 
SDSSiM ways in which the failure to give ex-slaves land created a cyde (of poverty) that led to 
sharecropping. For example: "no mobility or stuck and couldn't get North." 

Definition of Sharecropping (for a '2', can be implicit) 

- tax in kind 

- storeowner "loans" 
-racism 

- lack of ownership 

- description of how sharecropping came to be according to Douglass 
General Comments: NONE 
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Document £ - Charts and graphs showing economic information about the Great 
Depression 



BO001855 



UNEMPLOYMENT RATES, 1929--1941 
30- 




1 — 1 — I — I — I — r 



Years 



GROSS NATIONAL PRODUCT, 1920-1940 




1940 



THE FEDERAL BUDGET, 1929-1941 




oP oP ^ op ^ ^ ^ ^ 
Years 



6. Based on the graphs and your knowledge of history, you can conclude that 
the first successful steps toward partial recovery from the Great Depres- 
sion occurred 

® at the end of the First World War 
CD with the passage of the Hawley-Smoot Tariff 
CD with United States entry into the Second World War 
• with the election of Franklin Delano Roosevelt as President 



BOOOU56 
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12th-Grade Advanced 

Twelfth-^rade students achieving at the advanced level should demonstrate a 
comprehensive understanding of events and sources of U.S. history. Recognizing that 
Mstorv is subject to interpretation, they should be able to evalvate histoncal claims 
critically in light of the evidence. They should understand that important issues and 
themes have been addressed differently at different times and that America's politxcal, 
social, and cultural traditions have changed over time. They should be able to wnte well- 
reasoned arguments on complex histoncal topics and draw upon a wide range of sources 
to inform their conclusions. 



17, What 15 the mam message of this canoon? 



BOnC U6.J 
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U«m NunilK-r: 17 Accev:i;ion Number B(X)01464 



Wluit is the nujin inc5;j«ige of tliis cnrtoon? 

***inscn 5 Hncis*** 



Ratinnnle Text: 

1 - Iimppropriotii. 'Hie rtvspnnsc clues not correctly identify the message of the t;artoon. It does \M 

indicate thai the cartoon refers lo problems in the civj] rights movement in Uie United States. 

2 - Partial. The response understands that the cnrtoon is about problems associated with the civil rights 

but is not able to explain the cartotm in a specific and precise fashion. 

3 Appnipriate. The rr^T>onsc correctly identifies the main message of the cartoon as t^ing iha( passing 
civil rig!H<5 laws does not tncan iJiat civil rights issuevS are resolved, and that social, economic, and 
poliucnl (and not just legal) changes wciv also necessary. 

General Comments: NONK 
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Document F - Two people's memories of the eaily yeais of the Depression 



Oh; I remember having to move out of our home. My father had brought in a 
team of horses and wagon. We had always lived in that house, and we couldn't 
understand why we were moving out. When we got the other house, it was a 
worse house, a poor house. That must have been around 1934. 1 was about six 
years old. 

— Cesar Chavez 

The oil boom come |to Oklahom'i City] in '29. People come from every direction 
in there. A coupla years later, they was Uvin' in everything from pup tenu, houses 
built out of cardboard boxes and old pieces of metal that they'd pick up — anything 
that they could find to put somethin' together to put a wall around 'em to protect 
'em from the pubUc. 

— Mary Owsley 



9. What effect did the Depression have on famihes? Use both the passages and 
your knowledge of history in your answer. 

BO0C1S59 
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Item Number: 



Accession Number: BO001859 



Q3H1L09 



Document F - Two people's memories of the early years of the Depression. 

Oh, I remember having to move out of our house. My father had 
brought in a team of horses and wagon. We had always lived in that hours, and 
we couldn't understand why we were moving out. When we got the other 
house, it was a worse house, a poor house. That must have been around 1934. I 
was about six years old. 



The oil boom cam (to Oklahoma City) in '29. People come from every 
direction in there. A couple years later, they was livin' in everything from 
pup tents, houses built out of cardboard boxes and old pieces of metal they'd 
pick up anything that they could find to put somethin' together to put a wall 
around 'em to protect 'em from the public. 



What effect did the Depression have on families? Use both the passages and 
your knowledge of history in your answer. 



Rationale Text: 



!• - Inappropriate. The response does not indicate that the Depression led 
many famihes to become homeless or live in bad conditions, etc. OR, the 
responses are significantly exaggerated or inaccurate. 

2. . Partial. The response indicates that the Depression is associated with 

homelessness and led people to live in bad conditions. It does not use 
evidence in the passage or other historical data for support. OR, the 
response uses passages correctly but very letcrally fails to generalize 
from them. 

3. - Appropriate. The response indicates that the depressions dislocated and 

made many families homeless. The answer may also indicate that the 
Depression put huge tensions on many families. The answer refers to the 
passage or historical evidence in the response. The response shows a 
sense of the specific changes families were forced to undergo, and not 
just of bad conditions. 



General Comments: NONE 



- Cesar Chaves 



- Mary Owsley 
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1 

Figure 4a 

Understanding of Student Performance: Geography 
5 = Very Well Formed; 1 = Not Well Formed 




Basic Proficient Advanced 



Figure 4b 

Understanding of Student Performance: Histoi-y 
5 = Very Well Formed; 1 = Not Well Formed 



4.6 - 




Round 1 
Round 2 
Round 3 
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Figure 5a 

Conceptualization of Student Performance: Geography 
5 = Very Well Formed; 1 = Not Well Formed 
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Figure 5b 

Conceptualization of Student Performance: History 
5 = Very Well Formed; 1 - Not Well Formed 




i Hasic Proficient Advanced 
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Figure 6a 

Geography: Clarity of Rating Methods, by Rounds 
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Figiu-e 6b 

History: Clarity of Rating Methods, by Rounds 
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Figure 7a 

Geography: Ease of Applying Rating Methods, by Rounds 
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Figure 7b 

History: Ease of Applying Rating Methods, by Rounds 
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